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FOREWORD 


The Software Engineering Laboratory (SEL) is an organization 
sponsored by the National Aeronautics and Space Administra- 
tion/Goddard Space Flight Center (NASA/GSFC) and created for 
the purpose of investigating the effectiveness of software 
engineering technologies when applied to the development of 
applications software. The SEL was created in 1977 and has 
three primary organizational members: 

NASA/GSFC (Systems Development Branch) 

The University of Maryland (Computer Sciences Department) 

Computer Sciences Corporation (Systems Development 

Operation) 

The goals of the SEL are (1) to understand the software 
development process in the GSFC environment; (2) to measure 
the effect of various methodologies, tools, and models on 
this process; and (3) to identify and then to apply success- 
ful development practices. The activities, findings, and 
recommendations of the SEL are recorded in the Software 
Engineering Laboratory Series, a continuing series of 
reports that includes this document. The papers ' contained 
in this document appeared previously as indicated in each 
section . 

Single copies of this document can be obtained by writing to 

Systems Development Branch 

Code 552 

NASA/GSFC 

Greenbelt, Maryland 20771 
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SECTION 1 -INTRODUCTION 



SECTION 1 - INTRODUCTION 


This document is a collection of selected technical papers 
produced by participants in the Software Engineering Labora- 
tory (SEL) during the period December 1988, through October 
1989. The purpose of the document is to make available, in 
one reference, some results of SEL research that originally 
appeared in a number of different forums. This is the sev- 
enth such volume of technical papers produced by the SEL. 
Although these papers cover several topics related to soft- 
ware engineering, they do not encompass the entire scope of 
SEL activities and interests. Additional information ab.out 
the SEL and its research efforts may be obtained from the 
sources listed in the bibliography at the end of this docu- 
ment . 

For the convenience of this presentation, the seven papers 
contained here are grouped into three major categories: 

• Software Measurement and Technology Studies 

• Measurement Environment Studies 

• Ada Technology Studies 

The first category presents experimental research and eval- 
uation of software measurement and technology; the second 
presents studies on software environments pertaining to 
measurement. The last category represents Ada technology 
and includes research, development, and measurement studies. 

The SEL is actively working to increase its understanding 
and to improve the software development process at Goddard 
Space Flight Center (GSFC) . Future efforts will be docu- 
mented in additional volumes of the Collected Software Engi- 
neering Papers and other SEL publications. 
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SECTION 2-SOFTWARE MEASUREMENT AND 
TECHNOLOGY STUDIES 



SECTION 2 - SOFTWARE MEASUREMENT AND TECHNOLOGY STUDIES 


The technical papers included in this section were originally 
prepared as indicated below. 

• Establishing a Measurement Based Maintenance Im- 
provement Program: Lessons Learned in the SEL , 

H. Rombach and B. Ulery, University of Maryland, 
Technical Report TR-2252, May 1989 

• Maintenance = Reuse-Oriented Software Development , 

V. Basili, University of Maryland, Technical Report 
TR-2244 , May 1989 

• Software Development: A Paradigm for the Future , 

V. Basili, University of Maryland, Technical Report 
TR-2263 , June 1989 
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UMIACS— TR-89-54 
CS-TR-2252 


May, 1989 


Establishing a Measurement Based 
Maintenance Improvement Program: 
Lessons Learned in the SEL 

H. Dieter Rombacht 
Institute for Advanced Computer Studies 
Department of Computer Science 

Bradford T. Ulerv 
Department of Computer Science 
L'niversitv of Marvland 
College Park. MD 20742 


ABSTRACT 

The Software Engineering Laboratory (SEL) is a joint venture between NASA's 
Goddard Space Flight Center, the University of Maryland, and Computer Sciences 
Corporation. We discuss the use of a goal oriented approach to measurement to estab- 
lish a maintenance improvement program within the SEL. Differences are found to 
exist between the initial phase of the program and its routine application. We demon- 
strate our approach through concrete examples, and summarize lessons we have 
learned in the establishment of a measurement based, maintenance improvement pro- 
gram. 


t This work was supported by the NASA gram NS(ioI23 


This paper will also appear in the Proceedings of ihc Conference on SoUwarc Maintenance IPSO 
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1. INTRODUCTION 


The Software Engineering Laboratory (SEL) is an organization sponsored by the National 
Aeronautics and Space Administration/Goddard Space Flight Center (NASA/GSFC) and created 
for the purpose of investigating the effectiveness of software engineering technologies. The SEL 
was created in 1977 and has three primary organizational members: NASA/GSFC, the University 
of Maryland, and Computer Sciences Corporation. 

NASA/GSFC develops ground control software systems and other support software for 
satellites. A large number of case studies and controlled experiments have been conducted in the 
past that have resulted in evolutionary changes to NASA’s development practices 
iBasili85,McGarry85]. Some of the changes include the stricter use of code reading techniques 
Basili87], the use of measurement baselines for management purposes |Ramsey88j, measurement 
based recommendations for Ada projects [Brophy87|, and most recently the experimental 
adoption of the CLEANROOM development method iSelby87]. 

In order to formalize the procedures for investigating software technologies and to organize 
the resulting experience, three paradigms have been defined for goal oriented measurement: 

(1) Goal/question/metric Basili84,Basili85bi . This paradigm is based on the principle that 
effective measurement procedures should be derived (top-down) from goals. The GOM 
paradigm suggests that measurement needs to start with a precise specification of the goals, 
continue with the refinement of each goal into a set of quantifiable questions, and end with 
the derivation of a set of metrics. This approach yields a rationale for any chosen set of 
metrics all the way back to the original goals. Therefore, it also provides a basis for the 
goal oriented interpretation of collected data. 

(2) Evaluation [Basili84|. This paradigm is based on the additional principle that the 
measurement process must be designed to fit the production environment. The evaluation 
paradigm simply extends the GQM paradigm bv including the actual measurement 
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procedures. These procedures must be tailored to the product or process being studied in 
order to obtain valid results. Previous publications also refer to this as the GQM paradigm. 

(3) Improvement [Basili85b, Basili88] . This paradigm is based on the principle that 

improvement is based on continuous learning. The improvement paradigm provides the 
context for evaluating multiple projects. It emphasizes recording what has been learned 
through measurement so that this knowledge will be available when it is needed. 
Knowledge is managed explicitly by modeling the environment and providing feedback from 
analysis to production. 

Experience from several studies has reaffirmed these principles and our belief in the effectiveness 
of goal oriented measurement whether in a production environment or an experiment- This 
experience has led to the formulation of specific measurement guidelines and goal templates 
[Basili88j. 

A number of improvement programs have been established based on the improvement 
paradigm [Grady87,Rombach87,Basili85,McGarry85]. Most of the published results, however, 
address specific studies performed under such programs rather than the establishment of the 
programs themselves. Examples include industrial case studies 

Rombach87,Basili87b, Weiss85,Basili87j as well as academic experiments 
:Nehmer87,Katz86.Rombach86i. Two noteworthy exceptions address the managerial and 
technology transfer problems associated with the establishment of such a study 
[Brelsford88, Grady87b] . 

An organization’s long-term commitment to invest in such a program depends on whether 
the potential for future payoff can be demonstrated convincingly. We distinguish between the 
“initial program phase" aimed at establishing an effective program and demonstrating its payoff 
potential, and the “routine program phase'* aimed at applying an accepted program to routine 
projects. 
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The purpose of the initial phase is to understand the environment sufficiently to identify 
high leverage improvement goals and to establish a proper measurement infrastructure 
(procedures, managerial commitment, tools, personnel, etc,). The environment should be modeled 
explicitly and limited measurement may be required in order to demonstrate the potential 
leverage of the stated goals and the feasibility of the procedures. It is important to recognize that 
this initial phase represents an investment. What is learned in this phase provides the foundation 
for improvements during the routine phase. 

This paper reports on our experience from establishing an improvement program for 
maintenance in the SEL. Section 2 summarizes the approach used to create the initial 
understanding of the environment, the improvement goals and measurement procedures. Section 
3 characterizes the current status of our program. Sections 4 and 5 highlight some important 
lessons, and outline future SEL maintenance improvement activities. 


2. MAINTENANCE IMPROVEMENT APPROACH 

Within the SEL, development and maintenance are performed by separate organizational 
units. In late 1987. measurement of maintenance was included in the scope of the SEL in order to 
better understand and eventually improve the software life-cycle. At that time there was little 
documentation of the actual maintenance procedures (beyond some general guidelines) on which 
to base our initial analysis. Nor was there any explicit feedback to developers about the product 
maintainability, or the types and amounts of maintenance required. 

Based on past experience, we were confident that the guidelines supporting the improvement 
paradigm would be helpful during the initial program phase. We had, however, only vague ideas 
as to how these methods should be applied given the lack of explicitly documented experience in 
the SEL maintenance environment that was available when we began. We expected to learn 
about the strengths and weaknesses of the improvement approach in a situation where we could 
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not select improvement goals or design measurement procedures based on a mature understanding 
of the environment, but rather would have to initially bootstrap that understanding. 

The improvement program established for maintenance in the SEL is based on a version of 
the improvement paradigm applied to maintenance [Rombach88). This paradigm (see Fig. 1) 
suggests that maintenance can be improved by iterating the following steps for each project: (1) 
characterize the corporate maintenance environment; (2) state improvement goals in quantitative 
terms; (3) plan the appropriate maintenance and measurement procedures for the project at hand: 
(4) perform maintenance, measure, analyze and provide feedback; and (5) perform post mortem 
analysis and provide recommendations for future projects. 


11. Characterize the corporate maintenance environment 

12. State improvement goals 

a. State improvement goals informally 

b. Specify related measurement goals 

13. Plan maintenance 

a. Plan appropriate maintenance process 

b. Plan appropriate measurement process ; 

14. Perform maintenance 

a. Perform maintenance process ! 

i 

b. Perform measurement process 

c. Analyze collected data and provide immediate 
feedback 

15. Perform post-mortem analysis and provide recommendations 
for future projects 

16. Return to step II 


Figure 1: The improvement paradigm applied to maintenance. 


We applied the principles of the paradigms strictly. However, during the initial phase, our 
understanding of the environment, goals, and measurement procedures did not develop according 
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to a straightforward sequential application of the first three steps of the improvement paradigm. 
Nor were all supporting metrics identified by a strictly top-down application of the GQM 
paradigm. There are two good reasons for not following these steps: (i) we sometimes discover 
that our knowledge of prior steps is inadequate, so we retrace our steps, or (ii) practical 
constraints (such as existing data collection forms) preclude a strictly top-down derivation of 
procedures. 

The initial uncertainty in our understanding of the maintenance environment made it 
necessary to allow for planned and ad hoc feedback loops at any time. Such feedback loops 
resulted in revisions of the goals and measurement procedures. Measurement procedures were 
validated by actually applying them to real projects on a trial basis. The experience from such 
trial data collection, validation, and analysis helped us to further improve our understanding of 
the environment, and provided objective data to demonstrate the existence of suspected 
maintenance problems. Demonstrating the feasibility of planned measurement procedures on a 
trial basis has won confidence in their potential to support the improvement goals. 

In summary, a number of quick (and sometimes partial) iterations through the improvement 
paradigm eventually resulted in our current status. Based on this status the SEL has reached a 
consensus that routinely applying this improvement program to all future maintenance projects is 
worthwhile. 


3. PROGRAM STATUS 

This section presents the current status of our maintenance improvement program 
according to the outline of the improvement paradigm (Fig. 1). Section 3.1 summarizes our 
current understanding of the SEL maintenance environment (corresponding to improvement step 
II). Selected improvement goals and their supporting data collection, and validation procedures 
are summarized in sections 3.2. and 3.3 (corresponding to steps 12 and 13. b). Initial measurement 
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data, are presented and interpreted in sections 3.4 and 3.5 (corresponding to steps I4,b and U.c). 
This form of presentation is intended to demonstrate the use of the improvement paradigm 
during this initial phase. 


3.1. EINTVTRONMENT 

We characterise the SEL maintenance environment in terms of the application, maintained 
products, and maintenance process. 

Application 

Two missions in this study are the Cosmic Background Explorer (COBE) and the Gamma 
Ray Observatory (GRQ). They are tentatively scheduled to be launched in July, 1989, and April, 
1990, respectively. COBE’s scientific mission is to investigate the origins of the universe. GRO 
will make observations over the energy range from .1 to 30,000 MeV. 

The Earth Radiation Budget Satellite (ERBS) was launched from Space Shuttle Challenger 
in October, 1984. ERBS carried the Earth Radiation Budget Experiment (ERBE) and the 
Stratospheric Aerosol and Gas Experiment (SAGE)-H Measurements from these experiments are 
used to understand the earth’s climate and how environmental factors affect it. 

Largely because of the Challenger disaster in January, 1986, COBE will be the first mission 
to which the Flight Dynamics Division of NASA/GSFC has contributed significantly since ERBS. 

Maintained Products 

To date, we have monitored five projects representing each of the following three major 
types of system* developed in the SEL environment. 
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(1) Attitude Ground Support Systems (AGSS) provide operational support for a mission. Their 
functions include determining spacecraft attitude from telemetry data, verifying the on- 
board computer's attitude determination and control, supporting star tracking (for 
guidance), and more. 

(2) Attitude Telemetry Data Simulator Systems produce realistic attitude telemetry and 
engineering data files to exercise the algorithms and processing capabilities of AGSS s. 
Telemetry data includes essentially everything the spacecraft knows and could report back. 

(3) Attitude Dynamics Simulator Systems are analytic tools for testing and evaluating (two 
subsystems of) the spacecraft simulators. They simulate the environment of the spacecraft, 
sensor data, the on-board computer’s response (actuator commands), and the resulting 
control torques in order to model the spacecraft dynamics. 

In addition to the many numerical algorithms, each of these systems manages a user 
interface including the control of parameters, reading large data files, and printing tables and 
plots. 

, Maintenance Process 

In order to understand the role of maintenance in this environment, why changes occur and 
who could benefit from our observations, and in order to design effective measurement 
procedures, we have modeled the software life-cycle (Fig. 2). The Flight Dynamics Division of 
NASA/GSFC is divided into four branches, three of which are included in our model. Note that 
communications crossing organizational boundaries tend to be more formal and occur less 
frequently than internal communications. 

Project Requirements are received by the Specification Developers. The Specification 
Developers produce Functional Specifications and Mathematical Derivations for use hv the 
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Requirements Analysts. The Requirements .Analysts write the Preliminary Design which includes 
the high level system architecture. This design is used by the Software Developers who produce 
Code, a User s Guide, and a System Description. The System Description summarizes the 
Preliminary and Detailed Designs. The system is then submitted to Acceptance Testing. Upon 
acceptance, the system is eventually passed to Maintenance. Maintainers are responsible for the 
system until about three months after launch. If the system is an AGSS, its routine Operation is 
handled by Operations Support. Dynamics simulators and telemetry simulators are not passed on 
to Operations Support. Although changes frequently occur immediately after a launch, they are 
reportedly quite infrequent during Operational Support. 

During maintenance, each change is formally defined by an Operational Software 
Modification Report (OSMR), a form that specifies the change, and then follows it. gathering 
dates and signatures as the change is approved, implemented, tested, installed, etc. Typically 
there are more outstanding OSMRs than resources. A Project Task Leader is responsible for 
allocating these resources. 

OSMRs may be filed for several reasons. Acceptance Testing may reveal the need for 
enhancements (corrections are still the responsibility of the Software Developers). Later the 
Lsers (same organization unit as maintainers) may request enhancements or identify the need for 
corrections or adaptations. The Specification Developers may also initiate changes, resulting from 
ideas about similar forthcoming systems. Or, the Project Office may modify the Project 
Requirements. 

There are three software libraries: Working, Testing, and Operational. Entries in the 

Working library have not yet been accepted and may not be final. Several changes are made to 
the Testing library at once. These changes are tested together. The Operational library is very 
stable. Three months prior to launch, the Operational library is frozen. Maintenance 
nevertheless continues (the freeze is lifted after launch). 
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Figure 2. The software life-cycle in the NASA/GSFC Flight Dynamics 
Division. Organizational units are separated by the dotted lines. 
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Not all projects conform to the general models, but the models provide a common reference 
for tailoring procedures to specific projects and making comparisons or generalizations across 
projects. 


3.2. MAINTENANCE IMPROVEMENT GOALS 

We have identified a set of important improvement goals and refined them into quantifiable 
questions and metrics according to the GQM paradigm. This subsection presents some 
representative goals and questions. We have not included the complete set of goals, questions 
and metrics according to the GQM templates suggested in :Basili88i, but only highlight selected 
ones. In the questions that refer to specific metrics, the metrics are italicized. 

Characterizing maintenance 

We study the maintenance process itself to see how maintainers spend their time and what 
they do. We study the entire software life-cycle to understand how Specification Developers. 
Software Developers, Users, and Maintainers communicate; why changes are made; and whether 
the organizational divisions result in the best use of personnel's skills and knowledge. 

Currently we are interested in the following questions: 

(1) What are the major maintenance activities'? What are the major software life-cycle 
activities ? What are the major communication links between activities? 

(2) How is productivity related to types of changes (corrections, enhancements, adaptations), 
and characteristics of the product (type of product, LOC )? How is effort (hours) distributed 
across various activities? 
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Characterizing the delivered product 

The quality of the delivered product influences both what changes will be performed and the 
amount of effort that will be required. We therefore characterize the products with the following 
objectives in mind: to understand how and why the product changes; to understand how the 
product influences productivity during a change; and to provide historical, baseline data for 
future projects. 

We are currently interested in characterizing each of the three types of software products in 
terms of the following: 

(3) What are the static characteristics of each product (ULOC. 2 components , system 
architecture, programming language, types of documents )? What are the functional 
characteristics of each? 

(4) What types of changes are made (in terms of how both static and functional characteristics 
of the system are affected). 

Improving maintenance 

The maintenance process can be improved by focusing on the maintenance activities 
themselves, or by improving the entire software life-cycle of which maintenance is a part. 

We are currently interested in the following specific possibilities: 

(5) Establishing communication from Maintained to Software Developers for feedback about 
product maintainability. 

(6) Providing management with better mechanisms for monitoring the process. 

Improving the developed products 

Because of the relatively short maintenance phase, improvements to the products will be 
directed primarily toward the development of future products. 
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The following ideas have been proposed for improving the product: 

(7) The debug code is not well designed from the Maintained perspective. Future designs 
should allow the Maintainers more control over which messages are turned on. 

(8) We are trying to learn more about the relation between system structure and the locality of 
changes. A significant number of changes affect five or more files. 

3.3. DATA COLLECTION AND VALIDATION PROCEDURES 

The measurement procedures presented in this subsection support the stated improvement 
goals within the SEL maintenance environment. These procedures include data collection, their 
validation, analysis, and feedback. 

We routinely monitor the effort associated with various maintenance activities, and other 
characteristics of the changes. Similar data is available from development. This data will be 
used to characterize the maintenance process, the types of changes made to the product, and the 
reasons for making the changes. 

Routine data collection is implemented primarily through the use of forms (Fig. 3). At the 
end of each week, project personnel each complete a Weekly Maintenance Effort Form (WMEF) 
which briefly summarizes how they spent their time according to type of changes (correction, 
enhancement, adaptation, or other) and maintenance activity (isolation, implementation, unit 
test, integration test, other). Upon completion of each change, a Maintenance Change Report 
Form (MCRF, Fig. 4) is filed. The MCRF summarizes the change from a user’s perspective 
(reason for change and functionality) and from the programmer’s perspective (effort spent, parts 
of the system modified, etc.). A history of development (phase dates, effort) and product 
characteristics (size, number of subsystems, etc.) is summarized on a Project Completion 
Statistics Form (PCSF). This data will be made available at the end of development. It will be 
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compiled through the use of programs which examine the software library and the SEL database. 

The data collection forms and procedures reflect our models of the environment, the 
maintenance process and products, and measurement procedures. For example, the WTvfEF is 
filed only by Maintainers (as defined in Fig. 2); Specification Development and Use do not 
contribute to the hours monitored (although these hours are charged to the project); and 
Changed Objects (MCRF, section B) refers to specific documents used in this environment. 


Project Task 
Leader 

i 





MCRF 

SEL Library 

i 

1 \ 

WMEF 

(Form Validation) 

Programmers 



Figure 3, Routine data collection and validation procedures. WMEF is 
collected weekly from all project personnel (Project Task Leaders, Maintainers. 
Managers). MCRF is collected once for each change. These forms are validated, 
analyzed, and stored in the SEL, an independent entity. 
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original page is 

OF POOR QUALITY 


Name: 

MAINTENANCE CHANGE REPORT FORM 

OSMR Number: 

Project: 


Oat*: 



■ 

SECTION A: Change Request Information 

Funcdonal Peaertpdow of Changer 


What was the type of modification? 

What caused the change? 

Correction 

Requirernents/specfficat tons 

— Enhancement 

Software design 

Adaptation 

Coda 


Previous change 


_ Other 


SECTION B: Change Implementation Information 
Components Changed/Added/Deleted: 


1 hr to 1 day to 1 w*** to 
« ihr 1 day 1 watk 1 month >i 


Estimate the effort spent Isoiating/detemilning the change: 
Estimate the effort to design, implement, and test the change: 


Check all changed objects: 

Requlrements/Spectficattons Document 

— Design Document 
Code 

System Description 

User's Guide 

Other 


If code changed, characterize the change (check most 
applicable) 


» Logiccomrol structure 
(e.g„ changed flow of control} 

. Interface (Internal) 

(module to module communication ) 

. Interface (external) 

(module to external communication) 
i Data (value or structure} 

(e.g., variable or value changed) 
Computational 

(e.g M change of math expression) 
Other (none of the above apply) 


Estimate the rember of fines of code (including 

Enter the number of components: 

added 

Enter the number of the added components that 


changed deleted 


changed 


deleted 


totaOy new totally reused 


reused with 
modifications 


Figure 4. The MCRF. A change report summarizes the change from two 
perspectives: the functional perspective ("black box"), and the structural 

perspective ("white box"). 
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3.4. COLLECTED MAINTENANCE DATA 


We began monitoring maintenance projects on a trial basis in October, 1987 (Fig. 5). Our 
initial understanding of the environment reflected many biases from our knowledge of the 
Software Development process. Prior to October, 1988, the MCRF had emphasized corrective 
maintenance, did not request separate functional and structural descriptions of changes, and the 
OSMRs were not monitored. Other minor revisions were also made to the forms and procedures. 
The latest revision was made in January, 1989. Figure 6 summarizes the number of forms filed in 
total across the various projects. 


1987 

PROJl 

PROJ2 

PROJ3 

PROJ4 

PROJ5 


OND J FMAMJ 
c oe a e a .pau 
tvcnbrryn 


J 

¥ 


A S O N D 
u e c o e 
g p t v c 


J 

a 

n 


F 

p 

b 


1989 


VO 


VI 


V2 


Figure 5. Five maintenance projects have been monitored on a trial basis since 
October, 1987. Two changes to the data collection forms took place in October. 
1988, and January, 1989. 



PROJl 

PROJ2 

PROJ3 

PROJ4 

PROJ5 

MCRFs 

71 

18 

12 

0 

0 

WMEFs 

266 

86 

28 

15 

32 


Figure 0. Forms received. PROJ3 and PROJ4 have not been continuously 
active. 
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01/08/88 * 
01/15/88 * 

01 / 22/88 * 
01/29/88 
02/05/88 * 

02 / 12/88 * 

02/19/88 * 

02/26/88 * 
03/04/88 * 

03/11/88 * 

03/18/88 * 

03/25/88 * 

04/01/88 * 

04/08/88 * 

04/15/88 * 

04/22/88 * 

04/29/88 ** 

05/06/88 ** 

05/13/88 * 

05/20/88 * 

05/27/88 ** 

06/03/88 * 

06/10/88 * 
06/17/88 * 

06/24/88 * 

07/01/88 * 

07/08/88 * 

07/15/88 0 

07/22/88 0 

07/29/88 * 

08/05/88 0 

08/12/88 * 
08/19/88 * 

08/26/88 * 
09/02/88 
09/09/88 
09/16/88 
09/23/88 
09/30/88 
10/07/88 
10/14/88 
10/21/88 
10/28/88 
11/04/88 
DATE Nl 


* 


0 


* 

* 

* 


0 

0 

0 


0 


N2 


* 


* * 
* 


0 

0 

0 

0 

0 

0 

0 


* 

0 

0 

N3 


**** 

**** 

**** 

**** 

**** 

**** 

*** 

*** 

*** 

**** 

***** 

**** 

*** 

**** 

**** 

**** 

*** 

**** 


***** 

**** 

**** 

**** 

**** 

«** 

**** 

**** 

** 
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* * 

* * 

0 

*** 

** 


** 
** 
** 
* * 


** 

* * ** 
* * * * 
*** 
*** 
*** 
** 


0 

* * 

0 0 

* 0 

0 0 

*** 0 . 

P4 P5 


* 


**** 

**** 

** 

**** 
**** 
**** 
**** 
* *** 
* *** 
**** 
**** 
**** 
**** 
**** 
* 

*** 


* * ** 
* * * * 



* * * * 

* * * * 

* ** ** 
* * * * « 



0 

0 

0 


P6 


0 

0 

0 

0 

0 

P7 


0 


0 


MS 


0 =0 hours (form submitted) 

* = up to 10 hours 

** = up to 20 hours 

*** = up to 30 hours 

**** _ up to 40 hours 

***** __ U p to 5Q hours 


For each week of the project, total expended effort is shown distributed over the 
various personnel. On this project, Maintenance was contracted out to CSC 
(CSC manager: M8; CSC programmers: P4 - P7). Although NASA personnel 
(Nl - N3) were primarily responsible for Specification Development, some hours 
(meetings, consulting, etc.) were attributed to Maintenance. 


Figure 7. Weekly Effort Histograms (PROJl) 
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Real-time analysis of the data is not yet used by the Project Tasks Leaders or managers, 
but figure 7 suggests one way effort might be monitored. It shows for each week how much total 
effort has been invested by the various personnel. 

Figures 8 through 10 profile some overall trends. The FORTRAN subroutines in these 
products are not small, therefore entire components are seldom added or deleted. 


MCRF 

Total 

LOC 

LOC 

LOC 

Total 

Files 

Files 

Files 

= ===== 

LOC 

Added 

Deleted 

Changed 

Files 

Added 

Deleted 

Changed 

PROJl 

SIK 

2484 

430 

1353 

? 

3 

3 

335 

PROJ2 

46K 

2323 

325 

354 

433 

10 

0 

107 

PROJ3 

52K 

10 

0 

55 

242 

0 

0 

0 

PROJ4 

37K 

0 

0 

0 

322 

0 

0 

0 

PROJ5 

176K 

0 

0 

- 0 

? 

0 

0 

0 


Figure 8. This table summarizes data from section B of the MCRF. The totals 
refer to size at delivery ( ,,?M refers to unavailable PCSF data). Files (usually 
single subroutines) are called " components H on the MCRF. 


3.5. INITIAL ANALYSIS RESULTS 

In the initial phase of improvement, analysis results frequently reveal limitations in the 
measurement procedures and forms as weil as the naivete' of early goals and questions. 
Obviously, revealing these limitations and misconceptions is the first step toward improvement. 
The following examples demonstrate how one’s understanding and models develop through the 
analysis of data. These analyses are based on the current goals, questions, and data. 

(1) How expensive is maintenance compared to development? 

So far the costs have been low compared to figures often quoted in research literature (Fig. 
9). There are two very good reasons for this. First, maintenance, as defined in this environment. 
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does not include Operational Support (during which software changes are presumed infrequent). 
Second, excepting PROJl, none of the projects has completed. Also note that time spent in 
Specification Development (during the Maintenance phase) is not included. 


(2) What types of changes are made? 

Figure 10 shows the distribution of time spent on various types of maintenance over each of 
the projects. There are a few significant limitations to this data which make most generalizations 
premature: 1) the "other type" category was not included on the WMEF until October, 1988; 
time spent on meetings and management was therefore forced into one of the available categories; 
2) there is little total data from some projects; 3) PROJl was "maintained" by the original 
Software Developers; 4) there is no data summarizing those change requests which were not 
implemented. 



Development 

Hours 

Maintenance 

Hours 

PROJl 

17K 

3K ; 

PROJ2 

I8K 

2K 

PROJ3 | 

6K 

o 

to 

* 

PROJ4 

12K 

0 IK 

PROJ5 

47 K 

0.5K 


Figure 9. A comparison of total technical and management hours. (PROJ2 was 
in maintenance before Oct, 1987 when measurement of maintenance began.) 
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Total 

Correction 

Enhancement 

Adaptation 

Other* 

PROJl 

3286 

18% 

67% 

13% 

i% 

PROJ2 

2328 

44% 

54% 

0% 

0% 

PROJ3 

234 

58% 

35% 

3% 

3% 

PROJ4 

132 

0% 

87% 

3% 

10% 

PROJ5 

459 

10% 

78% 

2% 

10% 



Total 

Hours 

Isolation 

Change 
_ Design _ _ 

Implement 

Unit Test 

Integra- 
tion Test 

PROJl 

3286 

19% 

12% 

22% 

12% 

14% 

PROJ2 

2328 

15% 

15% 

19% 

6% 

30% 

PROJ3 

234 

53% 

12% 

23% 

6% 

4% 

PROJ4 

132 

45% 

15% 

24% 

15% 

0% 

PROJ5 

459 

15% 

10% 

15% 

4% 

46% 


Figure 10. These two tables show the distributions of effort (WMEF) bv type 
of change and activity. 

* “Other" was only recently added to section B of the WMEF. Most of the 
adaptation on PROJl actually represents management. 


4. LESSONS LEARNED 

The lessons learned from our efforts to establish the improvement program for SEL's 
maintenance environment address (i) why the introduction of measurement is significant, (ii) how 
well the improvement approach worked, (iii) how we built the credibility of our program, and (iv) 
automated support. 


4.1. WHY IS THE INTRODUCTION OF MEASUREMENT SIGNIFICANT? 

It was again apparent from this study that the introduction of measurement into an 
industrial setting represents not just the introduction of another method. Instead it signals a 
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dramatic change in the organization toward a more engineering oriented software development 
and maintenance style. Such a change affects all levels of an organization. Most organizations 
are not ready for this kind of change. As a consequence, the initial phase of the improvement 
program must be sensitive to the need of selling measurement as a credible and promising 
mechanism. 

Specific lessons learned: 

Ll: Introducing measurement represents a major shift toward a more engineering oriented 

software development and maintenance style. 

L2: Most environments are not ready for systematic, measurement based' improvement. 

L3: Special effort must be made to build the credibility of the selected improvement goals 

and measurement procedures before measurement is attempted on a large scale. 

The SEL adopted measurement as a means for routine improvement of its development 
activities over a decade ago. Still, during the introduction of measurement to the SEL s 
maintenance activities we encountered the need for selling measurement to a new audience. 
Initially, it was not clear whether we should aim to reduce the need for maintenance through 
better Specification Development, making a more maintainable product, or using more stringent 
acceptance testing. Our current goals were influenced as much by management's receptiveness to 
our ideas as by our technical understanding of the maintenance process. 


4.2. HOW WELL DED THE IMPROVEMENT APPROACH WORK DURING THE 
INITIAL PHASE? 

It became apparent again from this study that the basic principles of the improvement 
paradigm not only apply to the initial program phase; they are even more important for 
organizing learning the higher the level of uncertainty is. On the other hand, varying levels of 


20 


5642 


2-22 



understanding of the environment seem to justify (even require) different procedures for applying 
the paradigm. 

Specific lessons learned: 

L4: It is important to distinguish improvement methodologies (which depend on the 

environment and the maturity of the program) from principles of the improvement 

paradigm (which emphasize explicit learning through the use of measurement). As 

expected, the improvement paradigm was extremely helpful during the initial phase: 
however, the steps taken had to be modified according to the maturity of the program 
and the need to demonstrate its value. 

L5: Most initial learning came from exploratory investigations (e.g., meetings, interviews) and 

was based on subjective and intuitive data. 

In the initial phase, we learned more from attempting to implement the measurement 
procedures than from the data they provided. For example, many of the environment problems 
were first revealed to us in exploratory meetings with maintenance personnel. Those meetings 

provided the focus which enabled us to follow up on some of these issues in a much more goal 

oriented fashion (including the use of measurement data). Subjective data are very important 
during the initial program phase when our understanding does not allow for the definition of 
objective metrics, or when the underlying goal does not require or justify the cost of collecting 
objective data. 

Our application of the improvement paradigm can be characterized as prototyping. It 
allowed for feedback loops at any time. Sometimes our understanding of the environment was 
improved when refining goals into questions, metrics, and measurement procedures. An example 
of this is how we learned about the maintenance libraries. Given the use of formal change 
requests, we assumed that changes were well defined and that upon completion of a change, a 
corresponding change form (MCRF) would be filed. Eventually we learned that changes were 
being made (the hours showed on weekly forms), but we were not receiving MCRFs. This 
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inconsistency was only obvious when a new project started. "Completed" changes were being 
entered in the working library, but forms were not filed until after several changes were made, 
transferred to the test library, tested and approved. Although, we sometimes identified 
interesting metrics based on prior experience or intuition, we always eventually justified such 
metrics in the context of some improvement goal. 


4.3, HOW DID WE BUILD THE CREDIBILITY OF OUR PROGRAM? 

.Although the actual collection and interpretation of data was not the objective of the initial 
program phase, we used it on a trial basis in order to design effective measurement procedures 
and to identify further needs for improvement. It is very hard to convince anyone that you ar^ 
focusing on the right problems without actually providing some objective evidence that those 
problems actually exist. It is also difficult to convince someone that a program will be effective 
without demonstrating that the planned measurement procedures can be implemented in the 
current environment. The techniques used to establish procedures and demonstrate their 
credibility are important to the success of the program. 

Specific lessons learned: 

L6: The environment needs to be modeled at a level of detail that enables us to demonstrate 

that (i) the chosen goals are justified, (ii) the derived metrics support those goals, and (iii) 
the planned measurement procedures can be implemented in the given environment. 

Lw : Trial data collection and validation may be needed to establish confidence that planned 

measurement procedures are effective and that the identified goals address significant 
problems. 

L8: Depending on the initial level of understanding, data collection may be less accurate, 

objective, and complete during the initial phase than during routine application. 
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L9: The participation of all people concerned adds to the credibility of the program. 

We involved representatives from several organizational levels. They helped build initial 
hypotheses based on their insights, and served as reviewers of the results produced. Their 
comments were solicited, because without their confidence that we were addressing the right issues 
in a feasible way, we could not expect their cooperation during actual data collection. 


4.4. HOW MUCH AUTOMATED SUPPORT iS NECESSARY? 

This study also illustrated the need for automated support. During the initial phase, simple 
tools are needed for storing and analyzing the measurement data. A database system, statistics 
package, and report generator will become more important as the measurement procedures 
stabilize and the volume of data increases. Tools for modeling and planning could be very helpful 
provided they effectively support change. 

Specific lessons learned: 

LlO: Automated support for measurement is less important during the initial phase, but will 

be required during the routine phase. 

Lll: Graphical models ( c.g Fig. 2), plans (c.g., relating goals, questions, and metrics), and 

raw data are unwieldy and numerous. In addition to a database, automated support for 
designing and managing graphical structures would be extremely useful. 

During the initial phase, data collection forms were not stored in a database but simply in 
folders, and could not be analyzed automatically. As a result it was not always easy to keep 
track of forms and we might have lost some. This does not cause severe problems during the 
initial program phase, but would during the routine phase when reliance on this data is greater 
We are currently in the process of expanding the SEL development database to include 
maintenance data. 
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5. CONCLUSIONS 


In this study we have followed the principles of the improvement paradigm while 
introducing an improvement program to the SEL maintenance environment. We have observed 
that unlike a well conceived experiment and unlike an environment with a history of using 
measurement, this maintenance environment required a period of bootstrapping. Several rapid 
improvement (or learning) cycles were required to create the initial understanding of this 
environment necessary to identify meaningful goals and to design effective measurement 
procedures. Using measurement on a trial basis is also important for building the credibility of 
the improvement approach before attempting to apply it routinely on a large scale. 

We have completed the initial program phase in which the goals and measurement 
procedures presented in this paper have been demonstrated to justify routine application to all 
maintenance projects within the SEL. The SEL is now providing routine support for improving 
maintenance, including data collection, form validation, and database support. 

Establishing the SEL maintenance improvement program has been mainly a technology 
transfer problem. We tried to import existing technology, and customize it to the specific needs 
within the SEL. During the course of this study we have identified several problem areas that 
cannot be solved with existing technology but require additional research. These problem areas 
include more formal means for (i) capturing our understanding of an environment, (ii) packaging 
it into project specific, domain specific and general knowledge, (iii) relating measurement and the 
objects of measurement (i.e. processes and products), (iv) tailoring existing models to specific 
needs, and capturing the modeling process itself. Independent of the SEL maintenance 
improvement program, other research at the University of Maryland is addressing some of these 
issues: the TAME (Tailoring A Measurement Environment) project [Basili88| focuses on problems 
(ii) - (iv). The MVP (Multi-View Process Specification) project Rombach89i focuses on problems 
(i) and (iii). 
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ABSTRACT 

In this paper, we view maintenance as a reuse process. In this context, we dis- 
cuss a set of models that can be used to support the maintenance process. We present 
a high level reuse framework that characterizes the object of reuse, the process for 
adapting that object for its target application, and the reused object within its target ap- 
plication. Based upon this framework, we offer a qualitative comparison of the three 
maintenance process models with regard to their strengths and weaknesses and the cir- 
cumstances in which they are appropriate. To provide a more systematic, quantitative 
approach for evaluating the appropriateness of the particular maintenance model, we 
provide a measurement scheme, based upon the reuse framework, in the form of an or- 
ganized set of questions that need to be answered. To support the reuse perspective, a 
set of reuse enablers are discussed. 
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Abstract 

In this paper, we view maintenance as a reuse process. In this context, we discuss a set of models that 
can be used to support the maintenance process. We present a high level reuse framework that characterizes the 
• object of reuse, the process for adapting that object for its target application, and the reused object within its tar- 
get application. Based upon this framework, we offer a qualitative comparison of the three maintenance process 
models with regard to their strengths and weaknesses and the circumstances in which they are appropriate. To 
provide a more systematic, quantitative approach for evaluating the appropriateness of the particular maintenance 
model, we provide a measurement scheme, based upon the reuse framework, in the form of an organized set of 
questions that need to be answered. To support the reuse perspective, a set of reuse enablers are discussed. 


Introduction 

If we take the view that software should be developed with the goal of maximizing the reuse of prior 
experience in the form of knowledge, processes, products and tools, then the maintenance process is logically 
ideally suited to a reuse -oriented software development process. There are a variety of reuse models. The key 
issue here is which process model is best suited to the particular maintenance problem at hand. 

In this paper, we present a high level organizational paradigm for software development and maintenance 
in which an organization can learn from prior and current development and maintenance tasks and then apply 
that paradigm to several maintenance process models. The paradigm has associated with it a mechanism for set- 
ting goals that can be measured so that the organization can evaluate the process and the product and learn from 
its experience for future projects or enhancements of the current project. 

We begin by identifying three process models that can be used for maintenance. We then present a high 
level reuse framework that characterizes the object of reuse, the process for adapting that object for its target 
application, and the reuse object within its target application. Based upon this framework, we offer a qualitative 
comparison of the three maintenance process models with regard to their strenghts and weaknesses and the cir- 
cumstances in which they are appropriate. To provide a systemadc, quantitative approach for evaluating the 
appropriateness of the particular maintenance model, we provide a measurement scheme, using the 
Goal/Question/Metric Paradigm. Since reuse requires a supportive environemnt, a set of environmental reuse 
enablers are discussed. 


Maintenance 

The nature of software is that it can be modified without the use of physical tools such as screw drivers 
and soldering irons. This has lead to the false assumption that maintenance is easy and inexpensive. Clearly 
nothing could be further from the truth. 

Most software systems are complex and modification requires a deep understanding of the functional and 
non- functional requirements, the mapping of functions to system components, and the interaction of the 
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components. Without good documentation of the requirements, design and code with respect to function, tracea- 
bility and structure, maintenance becomes a difficult, expensive, error-prone task. As early as 1976, Beiady and 
Lehman reported on the problems with the evolution of OS 360 [7], The literature is filled with similar experi- 
ences and lessons learned [10,12,16,18,20]. 

Maintenance consists of several different types of activities: correction of faults existing in the system, 
the adaptation of the system to a changing operating environment, e.g., new terminals, operating system 
modifications, etc., and changes to the original requirements. The new system is like the old system but 
different in a specific set of characteristics. One can view the new version of the system as a modification of 
the old system or a new system which reuses many of the components of the old system. Although these two 
views have many aspects in common, they are quite different with respect to the process models used and their 
effects on future environments. 

In fact, we can identify at least three process models associated with maintenance depending upon the 
characteristics of the modification. We will call these (1) the quick fix model, (2) the iterative enhancement 
model, and (3) the full reuse model. All three models reuse the old system and so are reuse -oriented. Which 
model should be chosen for any particular modification is a combination ol management and technical decisions. 

Quick Fix Model. The quick fix model involves taking the existing system, usually just the code, and making 
the necessary changes to the source code and the accompanying documentation, e.g. requirements, design, and 
recompiling the system as a new version. This may be as straightforward as a change to some internal com- 
ponent, e.g. an error correction involving a single component or a structural change or even some functional 
enhancement Here reuse is implicit 


Old System New System 

Requirements Requirements <-- 

1 

Design Design < 

I 

Code > Code > 

I 

Test Test < 


Figure 1. Quick Fix Process Model 


Iterative Enhancement Model. Iterative Enhancement [5] is an evolutionary model which was proposed 
for software development in environments where the complete set of requirements for a system were not fully 
understood or the author did not know how to build the full system. Although it was proposed as a develop- 
ment model, it is well suited to maintenance. The process model involves: 

1. Starting with the existing system requirements, design, code, test and analysis documents 

2. Redeveloping starting with the appropriate document based upon analysis of the existing system, propagating 

the changes through the full set of documents 

3. At each step of the evolutionary process, continuing to redesign, based upon analysis. 
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Old System 

Requirements 
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Design 

I 

Code 

I 

Test 

I 

Analysis 


New System 

> Requirements > 

I I 

Design I 

I 1 

Code I 

I I 

Test I 

I I 

Analysis 


Figure 2. Iterative Enhancement Model 

To view this as a maintenance model, assume the initial implementation is the system in its current state 
in the evolutionary maintenance process. The process assumes that the maintenance organization has the ability 
to analyze the existing product, characterize the proposed set of modifications, and redesign the current version 
where necessary for the new capabilities. Again, reuse is implicit 

Full Reuse Process Model. While iterative enhancement starts with evaluating the existing system for 
redesign and modification, a full reuse process model starts with the requirements analysis and design of the new 
system, with the concept of reusing whatever requirements, design and code are available from the old system. 
The reuse process model involves: 

1. Starting the requirements f or the new system, reusing as much of the old system as feasible 

2. Building a new system using components from the old system or other systems available in the repository 

developing new components where appropriate. 


Old System 

Repository 


New System 

Requirements 

I 

— > ( Ri } < 

— > 

Requirements 

Design 

I 

— > { Di } < 

— > 

1 

Design 

i 

1 

PoHo _ _ 

___ .s / r* i i 


1 

Code 

w uc 
1 


> 

1 

Test 

> { Ti } < 

— > 

1 

Test 


Figure 3. Full Reuse Process Model. 

Here reuse is explicit, packaging of prior components is necessary and analysis is required for the selec- 
tion of the appropriate components. 
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The difference between the last two approaches is more one of perspective. The full reuse model frees the 
developer to design the solution relative to the available set of solutions of similar systems. The iterative 
enhancement model takes the last version of the current system and enhances it Both approaches encourage 
redesign, but the full reuse model suggests a wider forum and can lead to the development of more reusable 
components for future systems where the iterative enhancement model suggests the tailoring of an existing sys- 
tem for the given extensions. 


A Reuse Framework 

The existence of several models for maintenance raises several questions. Which is the most appropriate 
model for a particular environment? a particular system? a particular set of changes? the task at hand? How 
do I improve each step in the process model I have chosen? How do I minimize overall cost and maximize 
overall quality? 

In order to answer these questions we need a model of the object of reuse, the process of adapting that 
object for its target application, and the reused object within its target application. A simple model for reuse is 
given in figure 4. In that model, an object is any software process or product and a transformauon is the set of 
activities that are performed in reusing that object. Given that the scope of the new application are understood, 
the steps are: 

1 . identifying the candidate reusable pieces of the old object 

2. understanding them 

3. modifying them to our needs 

4. integrating them into the process 


* context * 

* I old I I new i * 

*| object ! >1 transformation j >1 object ! * 


**** | *★★****★( a****-******************-**-***-#****'******-*** 


I repository 1 


Figure 4. A Simple Reuse Model 


To flesh out the model, we need a framework for categorizing objects, transformations, and context. The 
framework should cover various categories e.g. reuse object: process, product Within each category there are 
various classification schemes for product: e.g. requirements document code module, test plan, and process: e.g. 
cost estimation, risk analysis, design. 

There are a variety of approaches to reuse and schemes that classify the object of reuse 
[8,9,11,13,15,17,19]. We use here a variation of a reuse framework [4] that captures several aspects of the reuse 
process, product and context 

Object dimensions include: 
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(1) Reuse Object Type: What is a characterization of the candidate reuse object? Sample categories and 
classifications are: process (e.g. design, test) and product (e.g. application, tool). 

(2) Self-Contained ness: How independent and understandable is the candidate reuse object? Sample categories 

and classifications are: syntactic independence (e.g. tightly coupled), semantic independence (e.g. similar 
functionality), and precision of specification (e.g. formal, informal). 

(3) Reuse Object Quality: How good is the candidate reuse object? Sample categories and classifications are: 

maturity (e.g. newly developed, used in one application) complexity (e.g. low cyclomatic complexity) reli- 
ability (e.g. no failures during prior use). 

Context dimensions include: 

(1) Requirements Domain: How similar are the requirements domains of the candidate reuse objects and 
current or future projects? Some example categories and classifications are: application (e.g. ground sup- 
port software for satellites), distance (e.g. same application, similar algorithms/different problem focus). 

(2) Solution Domain: How similar are the evoloution process which resulted in the candidate reuse objects and 

the ones used in the current and future projects? Some example categories and classifications are: process 
model (e.g. waterfall model) design method (e.g. function decomposition) programming language (e.g. 
FORTRAN). 

(3) Knowledge Transfer Mechanism: How is information about the candidate reuse objects and their context 
passed to current and future projects? An example classification is: humans (e.g. subset of the develop- 
ment team doing maintenance, separate team doing maintenance). 

Transformation dimensions include: 

(1) Type of Transoformations: How do we characterize the transformation activities to be performed? Some 

sample categories and classifications are: Percent of Change (e.g. 0%, 5%), Direction (e.g. general to 
domain specific, project specific to domain specific), mechanism of modification (e.g. verbatim, parameter- 
ization, template-based, unconstrained) and mechanism for identifying (e.g. by name, by functional require- 
ments). 

(2) Activity Integration: How do we integrate the transformation activities into the new system development? 

Some sample categories and classifications are: phase activity performed in the new development planning 
(e.g. cost estimation, risk analysis), construction (e.g. requirements development), analysis (e.g. testing). 

(3) Transformed Quality: What is the contribution of the reuse object in the context of the new system with 

respect to the objectives set for it? Sample category and classifications are: reliability (e.g. no failures asso- 
ciated with that component) and performance (e.g. satisfies the timing requirement). 

Comparing the Models Using the Reuse Framework 

When applying the reuse framework to the maintenance process, we are focusing on a set of reuse objects 
that are product documents. Let us compare the various models according to the dimensions given. 

Consider the reuse object dimensions: 

With regard to reuse object type, the object of the quick fix and iterative enhancement models is the set 
of documents representing the old system: The object of the full reuse model is the repository including the 
old system. 

With regard to seif-containedness, all the models depend upon the unit of change. The quick fix model 
depends upon how much evolution has taken place since entropy may have unstructured the system. In iterative 
enhancement, the evolved system should be improving for the specific application and for the appropriate set of 
changes, the unit of change should be more visible. In the full reuse model, the evolved system should be 
improving with respect to reuse object independence for the general application, depending upon the quality and 
maturity of the repository. 

With regard to reuse object quality, the quick fix model offers little knowledge of the quality of the old 
object In iterative enhancement the analysis phase provides a fair assessment of quality with respect to the 
particular application. In full reuse, we have an assessment of quality of the reuse object across several systems. 
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Consider the context dimensions: 

With regard to the requirements domain, the quick fix and iterative enhancement model assume the same 
application, in fact the same project. The full reuse model allows for manageable variation in the application 
domain, depending upon what is available in the repository. 

With regard to the solution domain, the quick fix model assumes the same solution structure exists during 
maintenance as during development. There is no change in the basic design or structure of the new system. In 
the iterative enhancement model, because redesign is a part of the model, there is some modification to the solu- 
tion structure allowed. The full reuse model allows major differences in the solution structure, i.e. a complete 
redesign is possible going from functional decomposition to object oriented design. 

With regard to knowledge transfer mechanism, the quick fix model and iterative enhancement work best 
with the same people. The full reuse model can compensate for having a different team, assuming we have 
application specialists and a well documented reuse object repository. 

Consider the transformation dimensions: 

With regard to type of activities, the quick fix model typically uses a source code look-up, reading for 
understanding, unconstrained modification and re-compilation approach. Iterative enhancement typically begins 
with a search through the highest relevant document, changing it and continuing through the subsequent docu- 
ments using a variety of modification mechanisms. The full reuse is uses a library search, and a variety of 
modification mechanisms depending upon the type of change. Here modification is done off-line. 

With regard to activity integration, in the quick fix model, all activities are performed at same time. Itera- 
tive enhancement associates the activities with all the normal development phases. In the full reuse model, 
identification or the candidate reusable pieces is done during project planning and the other activities are done 
during development. 

With regard to transformed quality, the quick fix model usually works best on small well -contained 
modifications since their affect on the system can be understood and verified in context. Iterative enhancement 
is more appropriate for larger changes where the analysis phase can provide better assessment of the full effect 
of changes. Full reuse is appropriate for large changes and major redesigns. Here, analysis and prior history of 
the performance of the reuse objects support quality. 

Given these differences, we can provide some analysis of the various maintenance process models and 
recommend where they might be most applicable. But first, let’s discuss the relationship between the develop- 
ment and maintenance process models. In some sense development can be considered a subset of maintenance. 
Maintenance environments differ from development environments with regard to the constraints on the solution, 
customer demand, timeliness of response, and organization. 

Most maintenance organizations are set up for the quick fix model but not for the iterative enhancement or 
reuse process models. This is because they are responding to timeliness, e.g. a system failure needs to be fixed 
immediately, or a customer demand, e.g, a modification of the functionality of the system. Clearly these are 
strengths for the quick fix model. But the weaknesses of the model are that the modification is usually a patch, 
not well documented, the structure of the system has been partly destroyed which makes future evolution of * he 
system difficult and error-ndden, and it is not compatible with development processes. This model is best used 
when timeliness and customer need are dominant and there is little chance the system will be modified again. 

The iterative enhancement model allows for redesign so the structure of the system evolves and future 
modification is easier. It focuses on the particular system, making it as good as possible. It is compatible with 
development process models. The drawbacks are that it is a more costly and possibly less timely approach (in 
the short run) than the quick fix model and it provides little support for generics or future similar systems. It is 
a good approach to use when the product will have a long life and evolve over time. In this case, if timeliness 
is also a constraint, the quick fix model can be used as a patch and the iterative enhancement model can be used 
for the long term change, replacing the patch. 

The full reuse process model provides the maintainer with a broader perspective, focuses on Jong range 
development for a set of products and has the side effect of creating reusable components of all kinds for future 
developments. It is compatible with development process models, and in fact, it is the way we would like such 
models to evolve. The drawback is that it is more costly in the short run, is not appropriate for small 
modifications but can be used in conjunction with other models. It is best used when we are living in mulu- 
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product environments or generic development where the product line has a long life. 

The assessment given above is informal and intuitive. This is due to the fact that it is a qualitative 
analysis. To do a quantitative analysis we need quantitative models of the reuse objects, transformations, and 
context. We need a measurement framework for characterizing via categorization and classification, evaluation, 
prediction, and motivation to support management and technical decisions. To do this we apply the 
goal/question/metric paradigm to the models. 


The Goal Question Metric Paradigm 

The goal/question/metric (GQM) paradigm [1,2,6] represents a systematic approach for setting the project 
goals (tailored to the specific needs of an organization), defining them in an operational, tractable way by 
refining them into a set of quantifiable questions that in turn imply a specific set of metrics and data for collec- 
tion. The tractability of this software engineering process allows the analysis of the collected data and com- 
puted metrics in the appropriate context of the questions and the original goal. This context supports feedback 
(by integrating analytic and constructive aspects) and learning (by defining the appropriate synthesis procedure 
for lower- level into higher- level pieces of experience). 

The process of setting goals and refining them into quantifiable questions is complex and requires experi- 
ence. In order to support this process, a set of templates for setting goals, and a set of guidelines for deriving 
questions and metrics has been developed [2]. 

Goals are defined in terms of purpose, perspective and environment. Different sets of guidelines exist for 
defining product-related and process-related questions. Product-related questions are formulated for the purpose 
of defining the product (e.g., physical attributes, cost, changes and defects, context), defining the quality perspec- 
tive of interest (e.g., reliability, user friendliness), and providing feedback from the particular quality perspective. 
Process-related questions are formulated for the purpose of defining the process (quality of use, domain of use), 
defining the quality perspective of interest (e.g., reduction of defects, cost effectiveness of use), and providing 
feedback from the particular quality perspective. 


Application of the Goal Question Metric Paradigm 

In applying the goal/question/metric paradigm, we define the goals of the maintenance process and articu- 
late the issues associated with choosing the appropriate process model, providing management with the questions 
that need to be answered to make intelligent decisions, understand the trade-offs, and perform nsk analysis. 
There are a variety of goals we can generate. For example: to determine which process model should be chosen 
for a particular product, to improve our performance or evolve a better definition of any of the models for a par- 
ticular product line. 

In what follows we will generate a sample goal for maintenance and provide a partial list of the quesuons 
involved. Some of the answers will be obvious, either in the measures they require be taken or the information 
required form the experts; others will not. Thus a goal for maintenance in the context of the reuse framework 
might be: 

Purpose: 

To evaluate the new product requirements in order to reuse as much of the available products as possible. 
Perspective: 

Examine the cost and future evolution of the development from the point of view of the organization. 
Environment: 

Along with the standard environmental factors, such as resource factor, problem factors, we would like to 
pay special attention to the three context dimensions of the reuse framework. 

Requirements Domain: 

Clearly we are using product objects from the same application domain, although we have the ability 
to choose candidate components from other application domains. 
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Solution Domain: 

This defines the process models, methods and tools that were used in the development of the existing 
product. If the same processes are to be used for the evolved project then there is not problem with 
reuse. However, the reuse model allows us to change the processes (and thus possibly to the product 
structure) at the cost of reusing less of the prior project If there arc to be changes then we must 
evaluate the cost of modification of the process and resulting product relative to the gains for process 
change. 

Knowledge Transfer Mechanism: 

If the maintenance group is the same as the development group then there is no transfer of knowledge 
required. If they arc different then there are concerns that must be evaluated with respect to applica- 
tion, process and product knowledge of the maintained and the kinds of documentation available. 

Product Definition: 

In considering the product, we actually have several. The new product to be built, i.e. the new version 
of the system, and the old versions plus any other systems that are relevant. 


Product Dimensions 


New Product* 

How many requirements are there in total for the new system? 

Old Product 

What is the mapping of requirements to system components? 

What is the measure of the complexity of the traceability ? 

How independent are the components to be modified? 

What is the complexity of the system and the individual system components? 

Repository: 

What candidate components are available in the repository and what are their context, transformation and 
object classifications? 

Difference between new and old: 

How many requirements are there that are not in the old system? (Categorize by size, new vs. modification 
of old vs. deletion of old, etc.) 

How many components must be changed, added, deleted? (categorized by size and type of change) 

Changes/Defects 

How many errors, faults, failures (categorized by class) are associated with the requirements and components 
that need to be changed? 

What is the profile of changes to the original system prior to this change? 

Cost 

What is the cost of understanding the new requirements? 

What is the estimated cost of building a new system, reusing the experience and parts of the old project? 

What was the cost of the old system in total? 

What was the cost of each version? 

What is the estimated cost of modifying the old system to meet the new requirements? 

Customer Context 


How will the new system be used? 

What are the potential future modifications based upon our analysis of customer profiles, past modifications and 
the state of technologies? 
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Perspective: 

cost and future evolution of the development 

Model of Perspective: cost of modification of the design of the system vs. the expected future modifications 
Parameters: 

the life time of the system 

the cost of future evolution of the system 

the cost of evolving the old system versus rebuilding from old parts 
Feedback: 

Is the model appropriate? 

How can the model be improved? 

How can the estimations be improved? 

How can classifications be improved? 

How can activities be improved? 


The GoaiyQuestion/Metric paradigm allows us to develop other goals for reuse. These can be developed 
for whether the reuse object is a process or a product Consider the following examples: 

Evaluate the modification activity within the reuse process in order to improve it. Examine the cost and correct- 
ness of the resulting object from the point of view of the customer. 

Predict the appropriate maintenance process model in order to perform the correct one. Examine its cost with 
respect to the customer needs and the future evolutions of the system from the point of view of the corporation. 

Evaluate the standard corporate design method in order to assess how it should have been tailored for the current 
project Examine its effectiveness from the point of view of the designer. 

Evaluate the components of the existing product in order to determine whether to reuse them. Examine their 
independence and functional appropriateness from the point of view of their use in future systems. 

Predict the ability of a set of code components to be integrated into the current system from the point of view of 
the developer. 

Motivate the development of a reusable set of components in order to engineer them for reuse. Examine the 
reward structure from the point of view of the manager and developer. 


Reuse Enablers 

There are a variety of support mechanisms necessary for achieving maximum reuse that have not been 
sufficiently emphaisized in the literature. In this paper we have discussed several of these: a set of maintenance 
models, a mechanism for choosing the appropriate such models based upon the goals and characteristics of the 
problem at hand, and a measurement and evaluation mechanism. To support these activities there is a need for 
an improvement paradigm that aids the organization in evaluating, learning and enhancing the software process 
and product, a reuse-oriented evolution environment that motivates and supports reuse, and automated support 
for that model as well as the measurement and evaluation process. 

The Improvement Paradigm: The improvement paradigm [1] is a high level organizational process model in 
which the organization learns how to improve their product and process. Within this model the organization 
should learn how to make better decisions on which process model to use for the maintenance of their future 
software products based upon learning from past performance. The paradigm is defined as follows: 

1. Planning. There are three integrated activities to planning that are iteratively applied: 

(a) Characterize the current project environment It provides a quantitative analysis of the environment and 
a model of the project in the context of that environment In the context of maintenance, the charac- 
terization should provide product dimension data, change and defect data, cost data and customer 
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context data for earlier versions of the system to be modified, information about what classes of candi- 
date components are available in the repository for the new system, and any information feedback 
from prior projects about experience with the different models for the types of modifications required. 

(b) Set up goals and refine them into quantifiable questions and metrics using the goal/question/metric para- 

digm, for successful project performance and improvement over previous project performances. This 
consists of a top-down analysis of goals that iteratively decomposes high-level goals into detailed sub- 
goals. The iteration terminates when it has produced sub-goals that we can measure directly. For 
maintenance this involves the development of specific G/Q/Ms as specified in the prior section. 

(c) Choose and tailor the appropriate construction model for this project and the supporting methods and 

tools to satisfy the project goals relative to the characterized environment Understanding the environ- 
ment quantitatively allows us to choose the appropriate process model and fine tune the methods and 
tools needed to be most effective. For example, knowing the effect of prior applications of the various 
maintenance models and methods in creating new projects from old systems allows us to choose and 
fine tune the appropriate process model and methods that have been historically most effective in 
creating new systems of the type required from older versions and component parts in the repository. 

2. Analysis. Analyze the data to evaluate the current practices, determine problems, record the findings and 

make recommendations for improvement. We must conduct data analysis during and after the project. The 
goal/question/metric paradigm provides traceability from goals to metrics and back. This permits the meas- 
urement to be interpreted in context ensuring a focused, simpler analysis. The goal-driven operauonal 
measures provide a framework for the kind of analysis needed. 

3. Learning and Feedback. This step involves the organization and encoding of the quantitative and qualitative 

experience gained from the current project into a corporate information base to help improve planning, 
development, and assessment for future projects. The results of the analysis and interpretation phase can be 
fed back to the organization to change the way it does business based upon explicitly determined successes 
and failures. In this way, we can learn how to improve quality and productivity, and how to improve 
definition and assessment of goals. We can start the next project armed with the experience gained from 
this and previous projects. For example, understanding the problems associated with each new version of a 
system, provides insights into the need for redesign and redevelopment. 

A Reuse -Oriented Environment; Reuse can be more effectively achieved within an environment that supports 
reuse [3,8,13]. Software engineering environments provide such things as a project data bases, and support 
the interaction of people with methods, tools and project data. However, experience is not controlled by 
the project data base or owned by the organization. Reuse only exists implicitly. 

We need to be able to incorporate the reuse process model into the context of development. We need to 
combine the development and maintenance models in order to maximize the context dimensions. We need to 
integrate characterization, evaluation, prediction and motivation into the process. We need to support learning 
and feedback to make reuse viable. We propose that the reuse model can exist within the context of the 
improvement paradigm, making it possible to support all of the above requirements. 

Th® TAME Project; The improvement paradigm and the reuse oriented process model require automated support 
for the data base, encoded experience, and the repository of prior projects and reusable components [2,3,14], 
We need to automate as much of the measurement process as possible, and provide a tool environment for 
managers and engineers to develop project specific goals, and generate operational definitions based upon these 
goals that specify the appropriate metrics needed for evaluation. The evaluation and feedback cannot be done in 
real time without automated support. Automated support will help in the post mortems analysis. 

The goal of the TAME system [2] is to instantiate and integrate the improvement and goal/question metric 
paradigms and help in the tailoring of the software development process. But it can also support the reuse- 
oriented process modeL The TAME environment model contains basic mechanisms for supporting systematic 
learning and reuse. To help with systematic learning it provides support for recording experience, off-line gen- 
eralizing or tailoring of experience, and formalizing experience. To help with systematic reuse it supports 
mechanisms for using existing experience and on-line generalizing or tailoring of candidate experience. In this 
way it attempts to integrate both learning and reuse into an overall evolution model. 
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The application of the TAME system concept to maintenance will provide a mechanism for choosing the 
appropriate maintenance process model for a particular project and provide data to help us learn how to do a 
better job of maintenance. 


Summary 

The approach to maintenance depends on the nature of the problem and the size and complexity of the 
modification. This paper recommends that we view maintenance as a reuse process. In this way the maintainer 
is provided with a reuse model and a framework for viewing maintenance that permits a measurement frame- 
work to be applied. A new model of a reuse-oriented evolution process can be developed in which the existing 
models can be defined. Existing models can then be analyzed within this framework, allowing an organizauon 
to evaluate the strengths and weaknesses of the different approaches and provides feedback in refining the vari- 
ous process models and creating an experience base from which to support further management and technical 
decisions. 

The approach provides support for defining activities, determining opdons, and evaluation. If the approach 
is not adapted then it is difficult for an organization to know which process model to use for a particular project, 
whether they are evolving the system appropriately, and whether they are maximizing quality and minimizing 
cost over the life of the system. 
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ABSTRACT 

This paper offers a new paradigm for software development that treats software 
development as an experimental activity. It provides built-in mechanisms for learning 
how to develop software better and reusing previous experience in the forms of 
knowledge, processes and products. It uses models and measures to aid in the tasks of 
characterization, evaluation and motivation. If proposes an organization scheme for 
separating the project-specific focus from the organization's learning and reuses 
focuses of software development. It discusses the implications of this approach for 
corporations, research and education and presents some research activities currently 
underway at the University of Maryland that support this approach. 
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1. INTRODUCTION 


We have been struggling with the problems of software development for 
many years [31,64]. Organizations have been clamoring for mechanisms to 
improve the quality and productivity of software. We have evolved from focus- 
ing on the project, e.g. schedule and resource allocation concerns, to focusing on 
the product, e.g. reliability and maintenance concerns, to focusing on the pro- 
cess, e.g. improved methods and process models [27,33,39,56]. We have begun to 
understand that software development is not an easy task. There is no simple 
set of rules and methods that work under all circumstances. We need to better 
understand the application, the environment in which we are developing pro- 
ducts. the processes we are using and the product characteristics required. 

For example, the application, environment, process and product associated 
with the development of a toaster and a spacecraft are quite different with 
respect to hardware engineering. No one would assume that the same educa- 
tional background and training, the same management and technical environ- 
ment, the same product characteristics and constraints, and the same processes, 
methods and technologies would be appropriate for both. They are also quite 
different with respect to software engineering. 

We have not fully accepted the need to understand the differences and learn 
from our experiences. We have been slow in building models of products and 
processes and people for software engineering even though we have such models 
for other engineering disciplines. Measurement and evaluation have only recently 
become mechanisms for defining, learning, and improving the software process 
and product [3.34]. 

We have not even delineated the differences between such terms as tech- 
nique, method, process and engineering. For the purpose of this paper we define 
a technique as a basic technology for constructing or assessing software, e.g.. 
reading or testing. We define a method as an organized management approach 
based upon applying some technique, e.g.. design inspections or test plans. We 
define a process model as an integrated set of methods that covers the life cycle, 
e.g., an iterative enhancement model using structured designs, design Inspections, 
etc. We define software engineering as the application and tailoring of tech- 
niques, methods and processes to the problem, project and organizational charac- 
teristics. 

There is a basically experimental nature to software development. We can 
draw analogies from disciplines like experimental physics and the social sciences. 
As such we need to treat software developments as experiments from which we 
can learn and improve the way in which we build software. 
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2. THE IMPROVEMENT PARADIGM 

Based upon our experiences in trying to evaluate and improve the quality in 
several organizations [5,29,53,58], we have concluded that a measurement and 
analysis program that extends through the entire life cycle is a necessity. Such a 
program requires an organization to adopt a long term, quality-oriented, organi- 
zational life cycle mode, which we call the Improvement Paradigm [4,19]. The 
paradigm has evolved over time, based upon experiences in applying it to 
improve various software related issues, e.g., quality and methodology. In its 
current form, it has four essential aspects: 

1 Characterizing the environment. This involves data that characterizes the 
resource usage, change and defect histories, product dimensions and environ- 
mental aspects for prior projects and predictions for the current project. It 
involves information about what processes, methods and techniques have 
been successful in the past on projects with these characteristics. It provides 
a quantitative analysis of the environment and a model of the project in the 
context of that environment. 

2 Planning. There are two integrated activities to planning that are itera- 
tively applied: 

(a) Defining goals for the software process and product operationally rela- 
tive to the customer, project, and organization. This consists of a 
top-down analysis of goals that iteratively decomposes high-level goals 
into detailed subgoals. The iteration terminates when it has produced 
subgoals that we can measure directly. This approach differs from the 
usual in that it defines goals relative to a specific project and organiza- 
tion from several perspectives. The customer, the developer, and the 
development manager all contribute to goal definition. It is, however, 
the explicit linkage between goals and measurement that distinguishes 
this approach. This not only defines what good is but provides a focus 
for what metrics are needed. 

(b) Choosing and tailoring the process model, methods, and tools to satisfv 
the project goals relative to the characterized environment. Under- 
standing the environment quantitatively allows us to choose the 
appropriate process model and fine tune the methods and tools needed 
to be most effective. For example, knowing prior defect histories 
allows us to choose and fine tune the appropriate constructive methods 
for preventing those defects during development (e.g. training in the 
application to prevent errors in the problem statement) and assessment 
methods that have been historically most effective in detecting those 
defects (e.g., reading by stepwise abstraction for interface faults). 

3 Analysis. We must conduct data analysis during and after the project. The 
information should be disseminated to the responsible organizations. The 
operational definitions of process and product goals provide traceability to 
metrics and back. This permits the measurement to be interpreted in con- 
text ensuring a focused, simpler analysis. The goal-driven operational meas- 
ures provide a framework for the kind of analysis needed. During project, 
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development, analysis can provide feedback to the current project in real 
time for corrective action. 

4 Learning and Feedback. The results of the analysis and interpretation phase 
can be fed back to the organization to change the way it does business based 
upon explicitly determined successes and failures. For example, understand- 
ing that we are allowing faults of omission to pass through the inspection 
process and be caught in system test provides explicit information on how 
we should modify the inspection process. Quantitative histories can improve 
that process. In this way. hard-won experience is propagated throughout 
the organization. We can learn how to improve quality and productivity, 
and how to improve definition and assessment of goals. This step involves 
the organization oi the encoded knowledge into an information repository or 
expereince base to help improve planning, development, and assessment. 


• Characterize the current project environment. 

• Set up goals and refine them into quantifiable questions and metrics for successful project 
performance and improvement over previous project performances. 

• Choose the appropriate software project execution model for this project and suppornmr 
methods and tools. 

• Execute the chosen processes and construct the products, collect the prescribed data, validate 
it, and and analyze the data to provide feedback in real-time for corrective action on Mie 
current project. 

• Analyze the data to evaluate the current practices, determine problems, record the findings 
and make recommendations for improvement for future projects. This is an off-line pro*’***.* 
which involves the structuring of experience so that it can be reused in the future. 

• Proceed to step 1 to start the next project, armed with the recorded, structured exn»Ti»*nce 
gamed from this and previous projects. 


FIGURE Is THE IMPROVEMENT PARADIGM 


The Improvement Paradigm is based upon the assumption that software 
product needs directly affect the processes used to develop and maintain the pro- 
duct. We must first specify our project and organizational goals and their 
achievement level. This specification helps determine our processes. In other 
words, we can’t define the processes and then determine how we are going to 
achieve and evaluate certain project characteristics. We must define the project 
goals explicitly and quantitatively and use them to drive the process. 

.As it stands, the improvement paradigm is a generic process whose stops 
need to be instantiated by various support mechanisms. It requires a mechanism 
lor defining operational goals and transforming them into metrics (step ’Ja). It 
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requires a mechanism for evaluating the measurement in the context of the goals 
(step 3). It requires a mechanism for feedback and learning (step 4). It requires 
a mechanism for storing experience so that it can be reused on other projects 
(steps 1,2b). It requires automated support for all of these mechanisms. In the 
next three sections, we will discuss mechanisms that have been used to support 
these activities. In the last half of the paper, we will discuss a proposed organi- 
zational structure that allows these activities to be managed and evolve. 


2.1. The Goal/Question/Metric Paradigm 

The Goal/Question/Metric (GQM) paradigm is a mechanism for defining 
and evaluating a set of operational goals, using measurement on a specific pro- 
ject. It represents a systematic approach for setting the project goals tailored to 
the specific needs of an organization, defining them in an operational, tractable 
way by refining them into a set of quantifiable questions that in turn implies a 
specific set of metrics and data for collection. It involves the development of 
data collection mechanisms, e.g.. forms, automated tools, the collection and vali- 
dation of data. It includes the analysis of the collected data and computed 
metrics in the appropriate context of the questions and the original goals. 


The GQM paradigm was originally developed for evaluating defects for a set 
of projects in the NASA/GSFC environment [28]. The application involved a set 
of case study experiments. It was then expanded to include various types of 
experimental approaches, including controlled experiments [4,22,25]. 

The process of setting goals and refining them into quantifiable questions is 
complex and requires experience. In order to support this process, i ser of tem- 
plates for setting goals, and a set of guidelines for deriving questions and metrics 
has been developed |I9|. These templates and guidelines reflect our experience 
from having applied the COM paradigm in a variety of environments. 

Goals are defined in terms of purpose, perspective and environment. Dif- 
ferent sets of guidelines exist for defining product-related and process-related 
questions. Product-related questions are formulated for the purpose of defining 
the product (e.g., physical attributes, cost, changes and defects, user context), 
defining the quality perspective of interest (e.g., functionality, reliability, user 
friendliness), and providing feedback from the particular quality perspective. 
Process-related questions are formulated for the purpose of defining the process 
(process conformance, domain conformance), defining the quality perspective of 
interest (e.g., reduction of defects, cost effectiveness of use), and providing feed- 
back from the particular quality perspective. 

The GQM provides a mechanism for supporting step 2(a) of the Improve- 
ment Paradigm which requires a mechanism for defining operational goals and 
transforming them into metrics that can be used for characterization, evaluation. 
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FIGURE 2: THE GOAL/QUESTION/METRIC PARADIGM 

prediction and motivation. It supports step 3 by helping to define the experi- 
mental context and providing mechanisms for the data collection, validation and 
analysis activities. It also supports step -4 by providing quantitative feedback on 
the achievement of goals. 

The GQM was originally used to define and evaluate goals for a particular 
project, in a particular environment. In the context of the Improvement Para- 
digm. the use of the GQM is expanded. Now, we can use it for long range cor- 
porate goal setting and evaluation. We can improve our evaluation of a project 
by analyzing it in the context of several other projects. We can expand our 
level of feedback and learning by defining the appropriate synthesis procedure for 
lower-level into higher-level pieces of experience. .-Vs part of the IP we can learn 
more about the definition and application of the GQM in a formal way, just as 
we would learn about any other experiences. 


2.2. The TAME Project 

The TAME project [18,19] recognizes the need to characterize, integrate and 
automate the various activities involved in instantiating the Improvement Para- 
digm, for use on projects. It delineates the steps performed by the project and 
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creates the idea of an experience base as the repository for what we have learned 
during prior developments. It recognizes the need for constructive and analytic 
activities and supports the tailoring of the software development process. 



FIGURE 3: THE TAME SYSTEM 


The TAME system offers an architecture for a software engineering environ- 
ment that supports the goal generation, measurement and evaluation activities. 
It is aimed at providing automated support for managers and engineers to 
develop project specific goals and specify the appropriate metrics needed for 
evaluation. It provides automated support for the evaluation and feedback on a 
particular project in real time as well as help prepare for post mortems. 

The Tame project was initiated to understand how to automate as much of 
the paradigm as possible using whatever current technology is available and to 
determine where research is needed. It provides a vehicle for defining the con- 
cepts in the paradigm more rigorously. 

A major goal for the TAME project is to create a corporate experience base 
which incorporates historical information across all projects with regard to 
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project, product and process data, packaged in such a way that it can be useful 
to future projects. This experience base would contain as a minimum the histori- 
cal data base of collected data and interpreted results, the collection of measured 
objects, such as project documents, and collection of measurement plans, such as 
GQM models for various projects. It should also contain combinations and syn- 
thesis of this information to support future software development and mainte- 
nance. 

TAME is an ambitious project. It is assumed it will evolve over time and 
that we will learn a great deal from formalizing the various aspects of the 
Improvement Paradigm as well as integrating the various sub-activities. It will 
result in a series of prototypes, the first of which is to build a simple evaluation 
environment. Building the various evolving prototypes and applying them in a 
variety of project environments should help us learn and test out ideas. 

Tame provides mechanisms for instantiating the Improvement Paradigm by 
providing an experience base to allow the storing of experience so that it i-nii be 
used on other projects (steps 1, 2a), further defining the various steps to bo per- 
formed (steps 1.2. 3, 4), and automating whatever is possible. 

3. A REUSE-ORIENTED SOFTWARE ENGINEERING MODEL 

The Improvement Paradigm, as instantiated in the TAME system, assumes 
that improvement can be achieved by iterating planning, execution of plans, and 
feedback across projects within an organization. Feedback can be viewed as 
reusing experience from the ongoing or prior projects to improve the planning or 
execution of ongoing or future projects. Learning can be viewed as the process of 
accumulating and packaging experience so it can be reused effectively. Thus. rh<* 
paradigm explicitly recognizes the need to capture and reuse knowledge, products 
and processes from prior projects. 

On the other hand, it should be noted that reuse can be an effWiivp 
mechanism only if it is paired with learning and viewed as an integral part of an 
improvement-oriented software evolution process model, [f we accept the fact 
that a better understanding of a process allows for more effective reuse. M reuse 
orientation” and ” improvement orientation” of a process model are identical 
attributes. Both are supported by experimentation. 

In a traditional software process model, learning and reuse only occur 
because of individual efforts or by accident. They are not explicitly supported 
and called out as desired characteristics of the development process. As a conse- 
quence, this experience is not owned by the organization (via the project data- 
base) but rather owned by individual human beings and lost after the project has 
been completed. A reuse-oriented process model must view reuse, learning and 
feedback as integral components, and place all experience, including software 
evolution methods and tools, under the control of ail experience base [20]. 
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Since improvement requires the feedback of available experience and feed- 
back is based on learning and reuse activities, a requirement for such a process 
model is that it support systematic learning and reuse. Systematic learning 
requires support for the off-line recording, generalizing or tailoring, and formaliz- 
ing of experience. Systematic reuse requires support for (re-)using existing 
experience. Off-line activities are performed independent of any particular pro- 
ject in order to improve the reuse potential of existing experience in the experi- 
ence base. 

Project goals are typically directed towards the development of a specific 
system. Thus olf-line activities must have their own organizational structure. 
They cannot be part of the normal development organization because they 
require a different focus, a different set of processes, and an independent cost 
base. 


For example, the objective of the recording process is to create a repository 
ot well specified and organized experience. It requires effective mechanisms for 
collecting, validating, storing and retrieving experience. This should not be part 
of the project focus. The project can contribute by making its experience avail- 
able to this independent organization, but cannot itself oversee the recording. It 
might not even be clear to the project what is worth recording. 

The objective of generalizing' existing experience prior to its reuse is to make 
a candidate reuse object useful in a larger set of potential target applications. 
The objective of tailoring existing experience prior to its potential reuse is to 
fine-tune a candidate reuse object to fit a specific task or exhibit special attri- 
butes, such as size or performance. Clearly a project cannot afford to generalize 
or tailor experience for another project within its budget constraints. Even 
worse, it may not have the perspective to do so since objectives and characteris- 
tics are dilferent from project to project, and even more so from environment fo 
environment. Generalizing and tailoring require a broader perspective of the 
organization and the products it develops. 

The objective of formalizing existing experience prior to its potential reuse is 
to encode it in more precise, better understood ways. Off-line tailored or gen- 
eralized experience needs to be formalized to increase its reuse potential and 
satisfy general reuse needs within an organization. The more we can formalize 
experience, the better it can be reused. 

Formalization activities include the movement from informal knowledge 
(e.g., concepts), to structured or schematized knowledge (e.g., methods), or even 
to completely formal knowledge or automation (e.g., tools). It requires models 
of the various reuse objects, notations for making the models more precise, nota- 
tions for abstracting reuse object characteristics, mechanisms for validating these 
models, and mechanisms for interpreting models in the appropriate context. 
Clearly the project has neither the budget nor the need to formalize its own 
experience. 
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Reuse requires a precise specification of the reuse context including the evo- 
lution process that is expected to enable reuse, and the characteristics of the 
available candidate reuse objects. The objective of a reuse-oriented software 
evolution process model is to support the use of previously accumulated experi- 
ence during such reuse activities as: (a) specifying reuse needs in a way that 
allows matching them with descriptions of available experience, (b) finding and 
understanding appropriate reuse candidates, (c) evaluating reuse candidates in 
order to pick the most promising candidate, (d) actually tailoring the reuse can- 
didate if necessary, (e) integrating the reuse candidate into the ongoing software 
project, and (f) evaluating the software project. 

A reuse-oriented software evolution environment is an integral part of the 
improvement paradigm. The mechanisms supplied by the TAME system to sup- 
port that paradigm are consistent with the mechanisms needed to support the 
reuse environment model with its experience base. It provides a mechanism for 
evaluating the recorded experience, helping us to decide what and how to reuse, 
tailor and analyze. It captures experience in the form of data from which models 
can be built to formalize experience. It supports continuous learning. 

It is clear that an experience base is a key component of the reuse and 
improvement paradigms. A project needs help in accessing the reusable experi- 
ence. If the experience is available (recorded), appropriate (tailored or general- 
ized), and well-packaged (formalized), it can be used by a project. But an 
experience base is more than a physical entity. It is an organization that must 
support all the off-line activities that support its creation and use. 


4. DIVIDING UP THE RESPONSIBILITIES AND ACTIVITIES 

Based upon the prior discussion, the implementation of the Improvement 
Paradigm would best be served by two separate and distinct organizational struc- 
tures. One organization is project-oriented. Its goal is to deliver the systems 
required by the customer. We will call this the Project Organization. The other 
organization, which we will call the Experience Factory, will have the role of 
monitoring and analyzing project developments, developing and packaging 
experience for reuse in the form of knowledge, processes, tools and products, and 
supplying it to the Project Organization upon request. The Experience Factory 
represents the experience base discussed above and the various activities associ- 
ated with building and modifying it, controlling its access, and interfacing to the 
Project Organization. 

Each project in a Project Organization can choose its process model based 
upon the characteristics of the project, taking advantage of prior experience with 
the various process models from the experience base in the Experience Factory. 
It can access information about prior system requirements and solutions, effective 
methods and tools and even available system components. Based upon access to 
this prior experience, the project, can choose and tailor the best possible process, 
methods and tools. It can reuse prior products tailored to its needs. 
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PROJECT ORGANIZATION 



EXPERIENCE FACTORY 


FIGURE 4: IMPROVEMENT AND REUSE 
ORIENTED-SOFTWARE ENGINEERING MODEL 

The Experience Factory analyzes the project development for all systems 
developed by the corporation. Based upon this it recognizes commonality among 
projects, generalizes knowledge and packages it for use across all projects. It 
creates a repository of reusable information. For example, it can develop resource 
models. _ defect models, and risk management models and tailor them for the 
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particular projects. It can develop processes, methods, techniques and tools and 
tailor them based upon the characteristics of the particular project. This can be 
accomplished based upon the Factory's analysis of the success and failure of the 
various activities across many projects. It can generate system components, at 
various levels of the architectural hierarchy based upon its recognition of com- 
monality. 


4.1. Some Specific Activities in the Project Organization 

Let us consider the activities of the Project Organization with regard to the 
development of a system and how it might use the Experience Factory while 
applying the improvement paradigm. 


At the start of a project, project management functions consist of activities 
such as resource and schedule planning, organizing, and staffing. These are 
covered by the characterizing and planning functions in the Improvement Para- 
digm. 

During the characterizing phase, based upon its needs and characteristics, 
the project can access the experience base for the information about similar pre- 
vious projects. This provides the project manager with a context for planning 
that includes resource estimation and allocation information, personnel experi- 
ence. software and hardware available for reuse, environmental characteristics of 
concern and sets of baselines for resources, schedules, defects, etc. The project 
can store information on its own characteristics back into the experience base for 
analysis. 

During the planning phase, the project can analyze prior goals ami use them 
as defined or tailor them (or have them tailored by the Component Factory.) for 
its needs. It can access the collection of construtive and analytic methods and 
toois, that have been effective and choose the appropriate ones that will help 
satisfy its goals. The goals and methods are influenced by the knowledge gained 
from the characterization phase, specifically with regard to elements of prior sys- 
tems that can be reused. These elements include data, such as baselines, process 
models that have been successful, including methods and techniques that have 
been tailored and tools that support those methods, and components of prior pro- 
jects such as requirements, design or code that can be adapted for the current 
project. The goals define the kinds of data that need to be collected as well as 
the mechanisms needed for collection. This provides the manager with informa- 
tion about what feedback will be provided for the project during development. 
The goals and process model, as tailored for the project, are stored in the experi- 
ence base for monitoring the current project and expanding the experience base 
for future projects. 
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PROJECT ORGANIZATION 
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FIGURE 5: ACCESSING THE EXPERIENCE FACTORY 

Project execution covers the directing and controlling activities as well as 
the development activities. 

During the execution phase, the project proceeds using the tailored process 
model, methods, techniques, and tools as specified in the planning phase. It uses 
prior product parts, supplied by the experience base. Feedback is supplied to 
project management to support directing and controlling of the project. During 
execution, project experiences, components and data are returned to the Experi- 
ence Factory and feedback is provided to the project. 


5642 


2-55 






- 14 - 


At project conclusion, the overall project is analyzed and the results are fed 
back to the project as well as packaged and incorporated into the experience base 
for use on future projects. 


4.2. Some Specific Activities in the Experience Factory 

The Experience Factory plays several roles. It builds and maintains the 
experience base, it interfaces with the project in the Project Organization by pro- 
viding information from the experience base and developing those elements that 
are requested by the project based upon its current level of expertise, e.g., 
tailored methods and tools and software components, and it acts as a quality 
assurance organization, providing feedback to the project with respect to its 
goals. .As such it has several process models associated with it. 


In building and maintaining the experience base, the Experience Factory 
performs the learning and reuse activities of recording, generalizing and tailoring, 
and formalizing. The degree to which it can perform these activities depends 
upon the breadth and depth of the information available and the level of tech- 
nology. 

It records information gathered from the various project developments. For 
example, it saves experiences from the projects it is monitoring, such as code 
modules, lessons learned on the project from the application of the constructive 
and analytic processes and measurement data, such as resource and defect data. 

It generalizes or tailors the information that it has gathered. For example, 
it uses the project-specific measurement data across several projects to create 
baselines such as detect profiles; it develops generic packages from project specific 
packages or instantiates a generic package for a specific project; it refines a 
design technology based on the lessons learned from applying it on a specific pro- 
ject: it parameterizes a cost model for a project or uses data from the project to 
improve the estimation capability of the model. 

It formalizes the information in the experience base to enhance its reuse 
potential. For example, it supplies code modules with their functional 
specifications and other appropriate documentation such as characterizing attri- 
butes, when needed; it makes more precise the steps in applying a method based 
upon lessons learned from its application; it builds cost models empirically based 
upon the data available; it develops management support systems based upon the 
available data and lessons learned; it builds automated support for methods. 

In responding to requests from a project, it provides whatever information it 
has available from the experience base and the people. The level of support 
clearly depends upon the state of the art in the packaging of experience. The 
interface with the Project Organization will change over time, starting with small 
packets of experience and building to higher level ones. 
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EXPERIENCE FACTORY 



FIGURE 6: ACTIVITIES IN THE EXPERIENCE FACTORY 

The actual information supplied depends upon the request and what is 
currently available in the experience base. For example, during characterization, 
it provides baselines and estimation models, and information on packaged pro- 
ducts, such as requirements templates or code modules. General defect baselines 
can be tailored to the specific project by limiting the projects considered to those 
with the same characteristics as the current project, e.g.. same application 
domain, same process model. 

During planning it supplies GQM models and process models, methods, tools 
and techniques. These can be obtained directly from the experience base or 
tailored for the needs of the project. For example, assuming that inspections are 
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chosen for the project and knowing the classes of faults found in similarly 
classified projects, the component factory might tailor the reading technology 
within inspections to concentrate on locating the kinds of faults that tend to 
occur in this type of project. They can also provide training and consulting on 
the use of the methods and models. 

During project execution, they can act as a contractor supplying various lev- 
els of project components. In fact from the Project Organization perspective, any 
component that can be well specified can be delivered by the Experience Factory. 
In turn, the Experience Factory can respond to the request by delivering an exist- 
ing component, modifying an existing component, e.g. instantiating a generic 
package from the experience base, or developing the component from scratch and 
adding it to the experience base. 

If we view quality assurance as the act of leading, teaching, and auditing the 
process, then it implies an organizational structure independent but interactive 
with the projects. (Note that this is different from quality control, which we 
define as the act of directing, influencing, verifying, and correcting the product, 
which implies a project controlled organization.) The Experience Factory is an 
ideal location for the quality assurance activities. 

In acting as a quality assurance organization, the Experience Factory audits 
activities and collects the prescribed data, provides feedback to the project in 
real time, and offers training in the various planning, constructive, and analytic 
approaches. The quality assurance activities is consistent with the activities of 
building and maintaining the experience base and responding to requests from 
the Project Organization. It also provides an independent chain of command and 
a corporate perspective with regard to goals, data collection, process and pro- 
ducts. 


4.3. Viewing the Experience Factory as a Component Factory 

As a particular dimension of the Project Organization and the Experience 
Factory, consider the activities of the Project Organization with regard to the 
development of a system and how it might use the Factory from the point of 
view of code development, e.g., as a Component Factory. We can view the pro- 
ject organization within the Project Organization as having the following activi- 
ties: 


Requirements Definition: The system analysts will interact with the custo- 
mer to determine project requirements. It is assumed that the analysts will know 
the application domain and what is available in the repository for reuse. They 
will have access to repository information about what kinds of components are 
available so they can make tradeoff decisions, negotiating with the customer for 
function vs. price. 
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Initially, this negotiation will be limited since the repository will be sparsely 
populated. This should change over time as the repository fills with com- 
ponents. It should be noted that the system analyst can use Factory components 
for building and analyzing prototypes of the system. 

Specification and Design: The requirements will be turned into a system 
design and specification for the required components. Those components that 
can be well specified can be turned over to the Experience Factory and orders 
will be filled for components. 

Initially, the specifications will be for low level components since the Factory 
will begin bottom up. .As time goes on and the repository builds up in terms of 
components, and the technology for recognizing, specifying and integrating larger 
pieces of systems develops, larger components can be ordered. 

The Experience Factory operates according to several process models. When 
an order for a component arrives, it can check its repository for the appropriate 
component or order it externally if it is available from an outside vendor. It tan 
develop it from scratch, using verification technology, based upon the fact, that it 
has the specification and the component it is developing is limited in size. How- 
ever, given that it has been required to deliver such a component, it can decide 
whether the component is of general use, from its knowledge of other projects, 
and can generalize or tailor the component, package it with the necessary attri- 
butes for future reuse and store it in the repository. 

.As an initializing activity, the Factory can analyze prior systems for reusable 
components and re-engineer them to seed the repository. It can develop com- 
ponents, so they are easy to combine, modify with respect to certain criteria and 
label and package appropriately. 

Integration and Evaluation: The project will have the task of integrating the 
components into its own specified design. These integrated components might l>r 
returned to the Factory for future use. It will then evaluate the system based 
upon the customer requirements and deliver the system. 


5. IMPLICATIONS OF THE NEW LIFE CYCLE ORGANIZATION 


5.1. Implications for Corporations 

One of the major problems with software development in the past has been 
that projects have been unable to explicitly reuse experience from prior projects 
or contribute to the experience base for future projects. This has been due in 
part to the fact that immediate project delivery goals and the more long-range 
goals of reuse and learning are distinct and not easily paired. Project schedule 
olten takes precedence over the luxtiry of passing on learned experience. 
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The new life cycle organization divides the focus of software development 
into two separate organizations. It separates the immediate project goals from 
the long range learning and reuse-oriented goals. In the approach, the Project 
Organization can focus on the customer needs and has the advantage ol access to 
a knowledgeable support organization in the form of the Experience Factory. 
The Experience Factory focuses on the organization’s goals to learn and reuse. It 
has the advantage of accumulating experience from a large number ot projects 
which provides it with a broader perspective than any particular project. 

This organizational structure has many advantages. It should promote 
higher quality and productivity because of reuse and learning. It can provide 
better and more focused education and training for developers and provide better 
methods and tools for them to use. 

It provides the corporation with a corporate asset in the guise of the Experi- 
ence Factory. The Experience Factory contains everything the organization has 
learned and developed that is useful for future developments as well as an assess- 
ment of the status of corporate quality and productivity. .As the Experience Fac- 
tory grows in its role and assets, the corporation can learn more and more from 
the various experiences across the corporation. 

There will be more emphasis on formalization of all parts ot management 
and development. Formal verification becomes cost effective since the correct 
units will be used in many systems; it becomes more applicable since we will be 
applying it to smaller units, at least in the beginning, where the technology is 
manageable. Formal models of risk assessment can be used since the experience 
base should provide a broad basis for understanding and comparison. 

The organizational scheme has the advantage that it can start small and 
expand with the growth in technology and the experience base. However, there 
are several issues that must be dealt with in putting this organization in place, 
e.g. financial and organizational. 

This organization requires separate cost centers for the Project Organization 
and the Experience Factory. There are several models of how the funding of the 
Experience Factory might work. For example, it could be funded out of cor- 
porate overhead which would grow with the success of the factory or projects 
could be billed for factory items. The right model will depend upon the com- 
pany and the organization and politics within that company. 

This organization requires a careful definition of management and responsi- 
bility structures. It is clear that we do not want to create new conflicts over 
responsibility for problems with packaged experience. 

This organization needs to be motivated and supported. Incentive and 
reward structures need to be developed. We will need to learn from experience 
gained from applying different financial and management structures. 
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5.2. Implications for Research 

There are several implications for research based upon this organizational 
structure. Many of the technologies already developed for programming in the 
small are applicable in the factory domain. For example, verification technology 
is already available for factory produced components and it is necessary and cost 
effective because those units will be reused many times. Research activities can 
focus on the transfer of these technologies. Therefore, user friendly tools to sup- 
port verification are needed. Based upon this formalization, we should learn 
more about the relevant primitives for particular application domains and how to 
encapsulate them. 

There are research activities associated with defining and tailoring models. 
These include process models, methods and tools: product models of the various 
products and qualities of those products; and models of information, like goal 
generation languages, cost, resource allocation, risk, and defect prediction. 
Models must be defined for the Project Organization and the Experience Factory 
and must take into account their interface. This involves the definition of 
languages for defining these models and tool generators, i.e. tools that can be 
instantiated to support variations of a method. 

There are research activities associated with generating larger product units 
from the Experience Factory. These include defining models of module intercon- 
nection languages that scale up, combining specifications and verifying them, and 
combining test plans to validate integrated components. 

There are research activities associated with the building and accessing of 
the experience base, e.g., mechanisms for encoding lessons learned into a model, 
tools for generating goals and mapping them onto measures, models that permit 
the model to learn automatically. 


5.3. Implications for Education 

The organizational scheme provides a focus for many of the technologies 
already taught at the University and so makes much of the current education 
more relevant. Topics that require more emphasis are formalisms of all kinds, 
e.g., verification technologies, formal requirements and specification notations, 
formal models of measurement and management. There is a need to teach stu- 
dents how to develop, use and assess methods and tools and deal with access and 
retrieval of libraries. Reuse and learning technologies need to be made available. 

There is a clear entry path for new software engineers through the Com- 
ponent Factory where they can develop small components under careful guidance 
and tool support and learn from the general experience base. As their experience 
grows they can be moved into any of the other higher level activities, e.g.. the 
Project Organization, or other parts of the Experience Factory. 
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6. RESEARCH ACTIVITIES AT MARYLAND THAT SUPPORT 
THE NEW LIFE CYCLE 

The paradigms and organization described in this paper offer a framework 
for research that focuses on the key issues for improving the software process and 
product in a context that permit the research to be used and experimented with 
in an industrial setting. Over the past dozen years, at the University of Mary- 
land. we have been working on several research projects whose goal is to evolve 
to this framework. 

The projects are organized into those dealing with the instantiation of the 
improvement paradigm in the SEL [5,46], where the concepts of the Project 
Organization and the Experience Factory have been evolving, the TAME project 
which is automating support for this framework in a formal way. and a variety 
of other projects which are attempting to understand, formalize and improve 
various process and product characteristics. 

A major source of activity has been the Software Engineering Laboratory 
(SEL). a joint venture of the NASA Goddard Space Flight Center, the L niversitv 
of Maryland, and Computer Sciences Corporation. The SEL has informally acted 
as an Experience Factory that supports project development. The application 
domain is ground support software for satellites. We have been building models 
and supplying these models and lessons learned back to projects so they can 
improve their process and product. This work has been performed via experi- 
ments of various kinds, dealing with resource, defect, process, and product 
models. 

In an attempt to better understand the environment we have used data col- 
lected during development to build various descriptive models of the SEL 
environment. In this way we have formalized knowledge from raw data to for- 
mal models or baselines and made the results available to the project organiza- 
tion for use in characterizing, planning and evaluating the project. 

We have collected data on resource expenditures, applied various existing 
models [32,47.49,61] and eventually built and tailored models that explicitly 
described resource allocation in the SEL environment [2.8,10.15.30], These are 
used lor estimating, planning and evaluating new projects. 

We have developed baselines for defects by accumulating defect data over 
many projects [62]. These defects a classified by phase and type. Thev vary 
with different project classifications [16]. They provide insight into the environ- 
ment, support for project management and evaluation, and point to areas areas 
that need improvement in the process [18], 

We have used various product metrics [41,45] to provide insight into the 
characteristics of the products being developed as well as evaluating the useful- 
ness of these metrics for the SEL environment [6,11.26,42], Areas of new technol- 
ogy that have been introduced, like Ada have generated the need for developing 
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new metrics to characterize new- product qualities [12,40). We have used these 
metrics as baselines to provide the project manager with insights as to what the 
problems may be* with the current development [38]. 

With regard to process improvement, we have built descriptive and perscrip- 
tive models of processes, methods and techniques and experimented with their 
application. The results of our studies are formalized and reused for future pro- 
jects within the limits of the technology available. 

In some cases, we have performed controlled experiments in which we 
analyzed the effects of various methods and techniques before recommending 
them on actual projects. We would then perform case study experiments to 
evaluate the effect of the method or technology on an actual project development 
to assure that it scales up and is applicable to the SEL environment. For exam- 
ple, we ran controlled experiments on a set of structured programming methods 
and techniques [17], various testing and reading techniques [23.54]. object 
oriented design in Ada [12,40] and the Cleanroom process model [59]. 

We then apply these approaches to projects within the project organization. 
We evaluate their effect there, and make recommendations, write lessons learned 
documents, and refine or change the models to incorporate what we have learned. 
In this way, the experiences gained from applying a particular model from the 
experience base is improved based upon the lessons learned from applying the 
model so that it can be used for future projects. Two case studies currently 
being run in the SEL, based upon controlled experiments, are the use of object 
oriented development in Ada [1,14,35] and the application of the Cleanroom pro 
cess. 


In other instances we have developed models and experimented directly on 
the projects. For example, we have evaluated the test methodology used tor 
acceptance test [51] and the methods used for maintenance [55] . 

Parts of the data collection process have been automated for the FORTRAN 
environment [37. 43] and are being automated for the transition to Ada 30] . 
Other tools have been developed that help support the various technologies used. 
Parts of the evaluation process have been automated using a knowledge base to 
create a decision support system [50,60]. 

The Tame project has focused on the architecture for the measurement and 
evaluation processes [19]. Work has been done by using studies performed in the 
SEL to define the process improvement mechanisms [18]. We have devised a 
resource planning and feedback model that is consistent with the Improvement 
Paradigm [43]. 

The Goal/Question/Metric Paradigm has been applied in a variety of 
environments other than the SEL and has evolved based upon these activities 
[29,53,58]. 
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We are currently working on supporting the automation of the generation of 
operational goals in a reasonably complete and consistent manner. A key aspect 
of the approach is that project personnel can generate goals that can be meas- 
ured and evaluated. We are working on extending the GOM templates into a 
goal generation language that will aid the goal writer in articulating questions 
and metrics based upon the goal and the model of the object of interest. We are 
currently experimenting with hypertext and attribute grammar technology to 
develop prototypes of this automated support mechanism. 

We are in various stages in the development of three measurement tools for 
analyzing programs in Ada and 0. A source code analyzer for various syntactic 
metrics, such as cvclomatic complexity and software science metrics, has been 
developed for Ada (ASAP) [38] and C (CSAP). A structural coverage analyzer 
(SCA) is under development for Ada [63). Data bindings analyzers are being 
developed for Ada and C based on prior versions of the tools for FORTRAN. 
SIMPL, and PL/C. 

We have developed a set of requirements and defined a system architecture 
for measurement tool generations using a parser generator that retains the parse 
tree for further transformations [4Sj. an enhancement of YACC and are experi- 
menting with the prototypes of this tool generation system. 

With regard to reuse, we have developed a model of a reuse support 
environment that can exist within the TAME framework [20j. We have applied 
the model to the maintenance process to show the advantages of viewing mainte- 
nance as a reuse process [7]. 

We are developing a model of reuse consistent with the approach presented 
in this paper that classifies the objects as they exist in the experience base, tin* 
reuse activities and the objects as they are reused [21 j. For example, the reusable 
object can be classified according to the characteristics of the unit itself, its inter- 
faces, and its context. The model recognizes the need to assess the qualities of 
the reusable object based upon the characteristics of the project in which it will 
be reused. 

We are working on a language and support system that takes elementary 
processes and generalizes them into more complex processes. Elementarv 
processes correspond to the " basic algorithms" used to perform small tasks, such 
as the addition of two atomic units. Our goal is to identify useful sets of elemen- 
tary processes, and then show how they can be combined and extended to per- 
form more complex actions (such as the addition of a stream of atomic units.) 
Using our language, abstract data structures may be mapped onto particular 
structures (e.g., the addition process for streams could be mapped onto a process 
for addition of arrays of numbers), and also composed with other structures (e.g.. 
an array addition task could be composed with a division task in order to create 
a module for computing means.) Finally the system will package resulting 
processes into an acceptable language component, whether a procedure or func- 
tion. Our current langague supports only functional processes, a future strp is n> 
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support the creation of data abstractions or modules. 

To study the issue of code reuse, the LASER project is currently building a 
system that examines exising systems in order to study and extract code that can 
be reused to seed a component repository. The system measures the various 
components in the system and identifies candidate reusable components based 
upon their lack of complexity, reusability within the existing system, indepen- 
dence, etc. These candidate components are then isolated (made independent) 
and qualified. The qualification involves the catagorization and classification 
based upon a number of attributes, and the association of a functional 
specification with the component. 

The approach expressed here provides a focus for further research issues. 
Some of the questions for which work has begun are: 

• How can process models be formally expressed so they can be communicated, 
analyzed and tailored? 

• How can various models be stored so they can be accessed by the GQM tool 
and help generate the automated collection of the appropriate measures? 

• How can a specific process model be developed that satisfies the definition and 
storage of the prior two questions? 

• How can we better capture and reuse experiences in the form of lessons learned 
from previous efforts? 

• What other measurement data can be automatically collected? 

• How could the set of measurement tools defined above be developed so that 
they can be tailored for various types of measures, maximizing the reuse of sys- 
tem components among the tools and the language independence? 

• How can we classify experience so it can be appropriately reused? 

• Based upon a specification, how can a component be devised quickly from ele- 
mentary processes? 

• How can we transform existing components to make them more independent, 
and measure the cost of reuse? 

• How can we have confidence that the factory-provided modules will do what 
we want? 

• How can we integrate aggregates of modules with their associated attributes so 
that they can be analyzed, managed, and controlled? 

• How can we verify properties of aggregates of modules, not just individual 
modules? 

• How can the test plans for components be combined to provide a test plan 
and oracle for aggregate application structures? 
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7 . CONCLUSIONS 

The approach expressed in this paper has evolved over years of studying and 
experimenting with software development and maintenance. It provides a com- 
patable and consistent framework for both software development and software 
engineering research. It recognizes and takes advantage of the experimental 
nature of software engineering. 

It allows us to understand how things are being done and where the prob- 
lems are by studying the process and product in actual environments. It allows us 
to formalize models of the process, product and knowledge. These models can 
then be analyzed. They can be used to form a basis for research and at the same 
time provide immediate input to project development. 

From a research perspective, it provides a focus for research problems based 
upon problems that need to be solved. It provides a framework to tie together 
existing pieces of research. 

From a corporate perspective, the approach can be applied directly and the 
organization can grow and build its own experience base. It supports technology 
transfer in a natural way and it ties the research and development organizations 
closer together. 
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ABSTRACT 


Software managers are interested in the quantitative management of software 
quality, cost and progress. There have been many of models and tools developed, but 
they are of limited scope. An integrated software management methodology, which 
can be applied throughout the software life cycle for any number purposes, is required. 

The TAME (Tailoring A Measurement Environment) methodology, developed at 
the University of Maryland, is based on the improvement paradigm and the 
Goal/Question/Metric (GQM) paradigm. This methodology helps generate a software 
engineering process and measurement environment based on the project characteristics. 

The SQMAR (Software Quality Measurement and Assurance Technology) 
developed in NEC is a software quality metric system and methodology applied to the 
development processes. It is based on the feed forward control principle. Quality tar- 
get setting is carried out before the Plan-Do-Check-Action activities are performed. 

These methodologies are integrated to realize goal-oriented measurement, process 
control and visual management. The Software Management Cycle is a substantiation 
of these concepts. Based on the TAME process model, development and management 
environments can be generated. The SQMAT system helps target setting, data analysis 
and visual display. 

In this paper we discuss a metric setting procedure based on the GQM paradigm, 
a management system called the Software Management Cycle (SMC), and its applica- 
tion to a case study based on NASA/SEL data. A method for evaluation Software 
Management Cycle process is described. The expected effects of SMC are quality im- 
provement, managerial cost reduction, accumulation and reuse of experience, and a 
highly visual management reporting system. 


KEYWORDS 

TAME, improvement paradigm, Goal/Question/Metric paradigm, SQMAR. 
Plan-Do-Check-Action activities, process control, visual management, software en- 
gineering process, goal-oriented measurement, software quality metrics. 
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1. Introduction 


Management plays a key role in the software development process. In the end, it is 
management's responsibility to produce and deliver a quality product productively and profitably 
and to generate corporate credibility with the customer. Thus, effective management methodolo- 
gies are needed to support management in assessing the current status of the project and achiev- 
ing delivery of the final system on-time, within budget, and with the specified product qualities. 
It would also be useful if the methodology supported the improvement of quality and productivity 
on the current project and on future projects. Many companies are working to provide such 
methods for their managers. 

However, it is difficult to assess the current status of a project precisely because of the lack of 
visibility of the software during development. It is even more difficult to predict project progress 
because of the lack of clearly defined goals, the lack of feedback in the achievement of those goals, 
and the difficulties caused by the variation in personnel. 


2. Supporting Methodologies 

Thus, requirements for the management methodology include the ability to make the software 
as visible, quantifiable and objective as possible. Several methodologies and paradigms use 
metrics to satisfy these management needs during development. There have been many software 
metrics proposed in the literature that attempt to provide the visibility, quantification and objec- 
tivity [Boeh76, McRW77, Muri80, BaKa83|. 

From a customer perspective of product quality, a comprehensive set of quantifiable software 
characteristics were proposed by Boehm, et al. [Boeh76] and later refined by McCall and Walters 
[McRW77j. Based on these studies, Software Quality Metrics (SQM) was developed by Murine 
(METRIQS Incorporated) as a quantitative software quality assessment technology (Muri80j. 

SQMAT 

Based upon the SQM, the NEC Corporation has developed a Software Quality Measurement 
and Assurance Technology (SQMAT) [AzSM87, AzSu86, SuAY85] and has been using it as one of 
the support tools in their software quality control (SWQC) group activities [Mizu82j. Quality 
control seminars are held periodically for every level of worker; programmer through general 
manager. The seminars are used to motivate as well as educate everyone with respect to the 
quality control technologies. 

SQMAT is a software quality metric system and methodology applied to the development 
processes, which takes experimental SQM results into consideration. SQMAT consists of a quality 
measurement and evaluation method with three levels of quality criteria, and a support tool for a 
visual display for management. Its most notable feature is that the feed forward control principle 
is employed in addition to the feedback control principle. That is, quality target setting is car- 
ried out before the Plan— Do-Check— Action activities (Deming’s PDCA cycle) are performed. 
SQMAT procedures are defined as follows: 

(1) In the TARGET phase, a quality priority ranking is established for the individual quality 
characteristics, based on the users’ requirements and the development policy. It is impor- 
tant to clarify the quality target, i.e., classify the quality characteristics into 3 categories 
and set the target quantitatively. 

(2) In the PLAN phase, Software Quality Measurement Criteria (SQMC), are set up and 
methods for achieving the target quality are discussed in advance, primarily with the 
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quality assurance people and managers. 

(3) In the DO phase, high quality software is produced by complying with development stan- 
dards and SQMC as guidelines. Before the formal review, the developer executes a quality 
self-check. 

(4) In the CHECK phase, the software is checked and evaluated against the individual quality 
criteria set up in the PLAN phase. Quality is measured by a third party. If errors are 
detected, problem reports are drawn up. After scoring, score sheets and quality graphs are 
developed, and the achieved quality is judged by comparing it to the target quality level. 

(5) In the ACTION phase, corrective action is taken, based on problem reports. Achieving the 
quality target permits proceeding on to the next phase. SQMAT can be applicable not only 
to large scale software, but also to small projects. NEC’s experience with the approach has 
had measurable results. For example, based upon comparison with historical data, (l) a 
number of errors have been eliminated during the design and implementation phases, and 

(2) productivity (measured by lines-of-source-code/hour) has increased by 10%. 

The Improvement Paradigm 

The Quality Improvement Paradigm [Bas85a] for software engineering processes is a top level 
paradigm that is based upon the scientific method as applied to software evaluation. It provides 
the view of software evolution as an experimental process from which we must learn and improve 
the current project as well as future projects (Characterize, Set Goals, Choose Methods, Build, 
Analyze, Learn and Feed Back). It is a meta-life cycle model that aims at improving the 
software quality and productivity based upon measurement and reuse of experience. It needs to 
be instantiated for a variety of sub-activities, e.g. specific processes such as testing, product 
reviews, managing. It consists of six major steps: 

(1) Characterize the current project environment. 

(2) Set up goals and refine them into quantifiable questions and metrics for successful project 
performance and improvement over previous project performances. 

(3) Choose the appropriate software project execution model for this project and supporting 
methods and tools. 

(4) Execute the chosen processes and construct the products, collect the prescribed data, vali- 
date it, and analyze the data to provide feedback in real-time for corrective action on the 
current project. 

(5) Amalyze the data to evaluate the current practices, determine problems, record the findings 
and make recommendations for improvement for future projects. 

(6) Package the experience in the form of updated and refined models and other forms of struc- 
tured knowledge gained from this and previous projects and proceed to step 1 to start the 
next project. 

This paradigm is aimed at providing a basis for corporate learning and improvement 
[BaRo87] and is based upon experience with measurement and evaluation of software development 
in a number of companies. 

Goal Quest ion/Metric Paradigm 

The Goal /Question /Me trie (GQM) paradigm [BaWe85, BaSe84] is a mechanism for generating 
measurement in a goal-directed manner. It represents a systematic approach for setting the pro- 
ject goals (tailored to the specific needs of an organization), defining them in an operational, 
tractable way by refining them into a set of quantifiable questions that in turn imply a specific set 
of metrics and data for collection (addresses the aspects related to step 2) of the improvement 
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paradigm). Appropriate metrics are tailored to each project based on the G/Q/M templates and 
past experience. It includes the development of data collection mechanisms, e.g., forms, 
automated tools, the collection and validation of data, and the analysis and interpretation of the 
collected data and computed metrics in the appropriate context of the questions and the original 
goals. 

In order to support the process of setting goals and refining them into quantifiable questions, 
a set of templates for setting goals, and a set of guidelines for deriving questions and metrics has 
been developed [BaRo88j. These templates and guidelines reflect our experience from having 
applied the GQM paradigm in a variety of environments (RoBa87, WeBa84, BaWe8l]. 

Goals are defined in terms of purpose, perspective and environment. Different sets of guide- 
lines exist for defining product-related and process- related questions. Product-related questions 
are formulated for the purpose of defining the product (e.g., physical attributes, cost, changes and 
defects, user context), defining the quality perspective of interest (e.g., functionality, reliability, 
user friendliness), and providing feedback from the particular quality perspective. Process- 
related questions are formulated for the purpose of defining the process (process conformance, 
domain conformance), defining the quality perspective of interest (e.g., reduction of defects, cost 
effectiveness of use), and providing feedback from the particular quality perspective. 

The TAME (Tailoring A Measurement Environments) system [BaRo88] is a measurement 
environment that supports and integrates the Quality Improvement and the Goal Question Metric 
paradigms. 

Based on the work at NEC, the TAME project, and the managerial requirements specified 
above, a management methodology, called the Software Management Cycle (SMC), has been 
developed. Its main concepts are goal oriented, process control and visual management. Manage- 
ment procedures, support tools and forms, and an evaluation method are provided as part of 
SMC. 


3. Relationship of the SQM, GQM, and SQMAT 

The SQM and the GQM are both mechanisms for measuring software quality. Both models 
are top-down and characterize quality characteristics at three levels. In the SQM, these levels are 
Factor, Criteria, and Metric. For example, a high level factor such as correctness is defined by 
the set of criteria traceability, completeness, and consistency which in turn are defined in terms of 
a predefined set of metrics. 

The GQM model consists of a goal, which is specified by a set of quantifiable questions, which 
in turn are defined by a set of metrics and data distributions tailored to the specific environment. 
Thus to define a high level goal like correctness of the final product, we must define a set of ques- 
tions that characterize the product (with respect to its physical attributes, cost of development, 
changes and defects, and customer base and operational profile), define a model for correctness 
(which could include such concepts as traceability, completeness, and consistency), provide 
insights into the validity of the model and the data within the particular environment, and the 
results of the model along with some possible substantiation of the model results. 

The SQM model predates the GQM model, but the latter is more general. The GQM can be 
used to characterize, evaluate, predict, or motivate a product, process, model or metric, with 
respect to a variety of perspectives (e.g. customer, developer, user, manager, etc.) based upon an 
open ended definition of quality. It takes into account the specific environment in which the pro- 
duct has been developed as well as assessment of such things as an evaluation of how well the 
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particular methods were used, how well the domain of application was understood in order to 
help interpret the resulting evaluation metrics appropriately. It also involves the feedback of 
information for future development through learning. 

The SQM model is written from the point of view of determining a set of quality characteris- 
tics of the final product from the point of view of the customer. It does not measure process for 
developing that product and since its viewpoint is that of the customer, it provides limited sup- 
port for learning, feedback and improvement within the development organization. Its measure- 
ment process tends to be passive and is not focussed on capturing the causes of the quality prob- 
lems. 

The measurement focus of SQM as used in SQMAT has evolved and widened over time and is 
currently more consistent with the GQM. This wider view of SQM uses metrics to measure qual- 
ity of an intermediate product from the point of user, developer and so on. 



GQM 

SQM 

narrow-sense 1 wide-sense 

Objective 

Characterize, Assess, 
Predict, Motivate 

Assess (Quality) 

Structure 

Goal 

Question 

Metric 

Factor 

Criteria 

Metric 

Usage 

Project & Quality Management 

Quality Management 

Object 

Any Product, Process, 
Model, or Metric 

Product 

.Any Product 
Process 

Viewpoint 

Developer, User, 
Manager, Corporate 

User 

(same as 
GQM) 

Establish- 
ment manner 
of GQM or 
SQM 

G 

Select or Tailor 

F 

Select 

Q 

Select or Tailor 

C 

Select 

M 

Select or Tailor 

M 

Select 

Select 
or Tailor 


Table 1. Features of GQM and SQM. 


4 


5642 


3-7 



4. Software Management Cycle (SMC) 

The Quality Improvement Paradigm provides a top level organizational perspective on the 
software development and maintenance process. SMC is the management procedure and support 
system under that paradigm. It emphasizes three concepts; goal— oriented measurement, process 
control, and visual management. In response to each concept, several activities are necessary. 
These activities, performed during the management procedure, make it possible for management 
to achieve higher quality and productivity. 

The management procedure used in SMC consists of the following five steps: 

(Step 1) Define system/project characteristics 

It is important to define the system characteristics in detail to reflect the user requirements 
for development. A set of system/project characteristics forms are prepared to gather informa- 
tion on the requirements and the current project status. 

This is equivalent to the first step of the Quality Improvement Paradigm. The system 
engineer is responsible for understanding the customer requirements for the particular project 
correctly. The development environment should be also clarified. This characterization permits 
the comparison of the current project with prior projects with similar characteristics. This infor- 
mation is used in the next step, 

(Step 2) Select Goals, Quest ions, and Metrics 

To achieve high quality and productivity, it is necessary to set the specific objectives. This is 
the key step to the success of the project. Unless the goals are appropriate, the project will fail. 
The GQM paradigm is used to do this. It satisfies the requirement for goal-oriented measure- 
ment. It helps both developers and managers clarify the objectives of the project prior to 
development. 

Guidelines and templates are used to establish the particular GQM used. Templates from 
prior systems can be used or modified for this project. For each metric, measurement instructions 
are prepared, which include the importance of metric, the collection method and person responsi- 
ble, the data presentation, the decisions affected etc. 

Besides the set of goals and metrics for the particular project, a common set of managerial 
metrics have been specified to be applied to all projects. We can gather the data for getting the 
level of quality and productivity through projects and development phases. The metrics from 
this common set are shown below. 

[ Quality ] 

- Number of detected errors at test phase (from 
integration test through system test) 

- Number of detected errors within six months after release 

[ Productivity ] 

- Number of specification pages 

- Number of non-comment source statement 

- Effort at each phase by man-hour 
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(Step 3) Select activities 


Methods to effectively achieve the objective are considered at this time. Appropriate activi- 
ties for economically producing the software and managing the quality of the project can now be 
chosen based on the specific objectives laid out in the GQM. 

This step is critical to achieve the objectives. Setting goals, without specifying the means to 
achieve them, is meaningless. Sufficient discussion on the activity selection process is necessary 
from various viewpoints; how they fit into the development environment, how they integrate with 
the management methods and training plans, etc. and how the help achieve the objectives, pro- 
vide focus for the questions and affect the definition of the metrics. 

For process control, a review checklist is prepared for each phase of development based on the 
metrics specified by the GQM model and past history, e.g. prior fault data. Feedback to the pro- 
cess should also be performed as soon as possible after a review. Problems can be easily found 
using the review checklist. Periodic checks; e.g. monthly, or at the final review of each develop- 
ment phase, are required to monitor the process. The earlier the phase at which monitoring 
starts, the more effective it is for quality improvement. Audit and configuration management are 
also process control methods governing quality. 

(Step 4) Measure and assess the process and the products 

Project data will be collected periodically, at least at the end of each development phase. 
Based on the metrics selected, the process and the products are measured. The results are 
assessed by using specific rating criteria. It is helpful for manager to take proper action quickly. 
Continuous measurement and assessment can produce high quality product. 

For visual management, graphical displays of the appropriate management information can 
be selected based on the graph selection form. The project’s current status can be found by using 
the visual display tool provided by the SMC system. It is helpful for software managers to see 
the achieved quality level in a concrete form to support such activities as decision making, the 
management of quality and scheduling of the next workload. For example, graphs provide the 
manager with time series data indicating process and product changes, as well as comparative 
data from past projects. 

(Step 5) Support corrective action 

For low scoring metrics, some action should be taken. A corrective action list is prepared and 
used to improve both the current and future process and products. 

Based on the assessment of results at step 4, proper action is required quickly for problems or 
the sign of any problems. If necessary, the activity plan can be revised. These experiences are 
accumulated and used to future projects. 


Guidelines necessary to perform project management, based on SMC, are as follows: 


- Goal selection 

- Question selection 

- Metric selection 

- Activity selection 


- Management graph selection 

- Project status diagnosis 

- Corrective action recommendation 

- Reliability prediction 
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The SMC helps the manager in the definition of an appropriate software engineering process 
during the GQM and activities selection phases (steps 2 and 3), by allowing the manager to tailor 
goals, measures, methods and tools to the specific system/project characteristics. A data base can 
be defined and built to support the measurement environment during the GQM selection phase 
and to support both the development and management environments during the activities selec- 
tion phase. After executing one whole cycle through the SMC process, the results of analyzing the 
current project data can be fed back to each SMC phase. Updating the database and improving 
each step of the SMC helps generate a software engineering process for future projects. 

The SMC support system is currently a prototype built on top of existing software packages. 
It consists of (1) a data base, (2) a set of statistical packages, and (3) a set of graphical types 
(developed using Microsoft Excel), all integrated under a common user interface. 

Accumulation of application information in a data base enables the organization to establish 
guidelines for future projects. Therefore, the relation between the system characteristics and the 
measurements associated with the particular GQM should be collected and saved in a data base. 
Emphasis should be on the metrics common across several projects 


5. An example G/Q/M 

A simplified pair of GQM models, one for product and one for process are given. They are 
written from the point of view of the manager (which may include some of the concerns of the 
customer) for evaluating various components to improve quality, cost and usage of methods based 
upon managerial data. 

First we will define some terms and offer a model of the qualities of interest: 

DEFINITIONS: 

Size (NCSS) = the number of non-commentary source statement (NCSS) 

Actual Effort (AEF) = total number of staff hours to develop a component 

Estimated Effort (EEF) — estimated number of staff hours based upon the software science 
metric, E 

Actual Errors (AER) = the total number of errors reported 

Estimated Errors (EER) = the estimated number of errors based upon the software science 
metric, B 

Actual Error Rate (AERR) = AER / NCSS 
Estimated Error Rate (EERR) = EER / NCSS 
Changes (CH) = the total number of changes reported 
Change Rate (CR) = CH / NCSS 

Effort Distribution (PED) = the percent of staff hours for a particular component spent in each 
phase 
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Test Efficiency (PTE) = the percent of machine time spent testing a component 
Work Rate (WR) = NCSS / AEF 
Effort Variance (EFV) = AEF / EEF 
Error Variance (ERV) = AER / EER 


MODEL: 

The objectives for management are cost, quality and the effectiveness of the methods. 
Evaluation is performed on the basis of improvement over some norm. 

Cost can be assessed as the relationship between input, staff effort, and output, the quantity 
of documentation and program produced. In this case we will consider cost as demonstrated by 
two factors: work rate (WR), which provides some measure of the cost of production for a line of 
code, and Effort Variance (EFV), which provides some measure of whether the effort is reasonable 
relative to some measure of the expected effort. 

Quality is assessed in two categories, must-be quality and attractive quality. These terms, 
must-be quality and attractive quality, are common Japanese quality perspectives. Must-be 
quality means the fundamental qualities necessary for software to function, i.e., functionality and 
reliability. Attractive quality means any additional quality characteristics for the software to 
satisfy the users specific needs, e.g., usability, security, portability. In this case, we will consider 
quality as demonstrated by two factors: error variance (ERV), which provides some measure of 
whether the error rate is reasonable relative to some measure of expected errors, and change rate 
(CR), which provides some measure of the entropy of the system. 

Method characteristics are assessed based upon their adherence to a set of standards. Project 
manager experience is also assessed since the success of a project deeply depends on his ability. In 
this case, we consider method evaluation using two factors: effort distribution (PED), which will 
provides us some insight into whether the distribution of the effort was acceptable according to 
standard baselines of effort distribution, and test efficiency (PTE) which when combined with test 
time, will provide some insight into the effectiveness of the test process, and therefore the effec- 
tiveness of the methods used for development. 

Note that the model uses the software science measures, E and B as a basis for estimating, 
effort and bugs. It assumes these calculated values as basic estimates for the variables effort and 
errors and uses them as norms when comparing the actual values for effort and errors. 

In our proposed model, the values of these variables for any component are then compared to 
the values for some normal population. All values within 2 sigma variation from the average are 
considered acceptable. Those values with more than a 2 sigma variation in the "right" direction 
are considered good; those with more than a two sigma variation in the "wrong" direction are 
considered as not meeting the target goal. For example, the effort variance (EFV) for a com- 
ponent is considered bad if it is greater than two sigma above the norm determined by the aver- 
age value of cost for the rest of the population. 

In the example given in the next section the baselines are determined by the the rest of the 
component population in the particular project. In an environment where there is data from a 
sufficient number of projects, the baselines could be determined by projects with similar charac- 
teristics from other projects. 
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PRODUCT GOAL: 


Purpose. Evaluate various software components within a project in order to assess them and 
recommend areas for improvement. 

Perspective: Examine the relative cost and quality from the point of view of the manager. 
PRODUCT DEFINITION: 

Product Dimensions: A quantitative characterization of the physical attributes of the product. 

QI. What is the size of each component in terms of non-commented source statements (NCSS)? 
Q2. What is the value of the software science metrics for each component (E,B)? 

Changes/Defects: A quantitative characterization of the enhancements, errors, faults, and failures. 
Q3. What is the number of defects associated with each component (AER)? 

Q4. What is the number of changes associated with each component (CH)? 

Q5. What is the fault rate, change rate (AERR, CR)? 

Cost: A quantitative characterization of the resources expended. 

Q6. What is the staff effort involved in the development of each component, i.e. design, code 
test? 

Q7. What is the distribution of effort spent in the design, code and test phase (PED)? 

Context: A quantitative characterization of the customer community and their operational 
profiles. 

(No questions for this example] 

In general, five viewpoints are necessary for process questions. Two of five, "Effort of Use" 
and “Effect of Use", are actually used in a case study next section. 

PROCESS GOAL: 

Purpose: Evaluate the design, code and test processes in order to improve them. 

Perspective. Examine the relative cost distribution and test efficiency from the point of view of 
the manager. 

PROCESS QUESTIONS: 

Quality of Use. A quantitative characterization of the process and an assessment of how well it is 
performed. 

Q8. How much experience does the team have with respect to the methods and tools used? 

Q9. How much experience does the manager have with respect to similar projects? 
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Domain of Use: A quantitative characterization of the object to which the process is applied and 
an analysis of the process performer’s knowledge concerning this object. 

Q10. How understandable are the requirements? 

Effort of Use: A quantitative specification of the quality perspective of interest. In this case, a 
quantitative specification of the costs. 

Q6. What is the staff effort involved in the development of each component, i.e. design, code, 
test? 

Q7. What is the distribution of effort spent in the design, code and test phase (PED)? 

QU. What is the machine time spent in the test phase for each component (PTE)? 

Feedback from Use: This includes questions related to improving the process relative to the qual- 
ity perspective of interest. 

Q12. What is the input to the design and code methods and tools, and the defect detection 
methods and tools? 

Q13. What should be automated? 


6, Case Study 

The concepts of SMC can be applied to a variety of project types because of the flexibility of 
this methodology. Metrics and development methodologies are tailored to each project. In this 
section, we discuss several general issues in applying SMC and provide a sample application to the 
management of a specific project based upon models and the goals, questions and metrics of the 
previous section. 

In executing SMC in a project, the software management procedure mentioned previously, the 
templates, guidelines and some forms are used. A step by step approach based on this procedure 
is demonstrated. A sufficient budget for managing these activities is required. It is also neces- 
sary to establish an organization to support the SQM process. Certainly, a seminar on SMC for 
both managers and developers would have provided better results. It should be remembered that 
the more experience the manager and the organization have with SMC, the better they will be 
able to apply the method. The continuous application of the method provides a better support 
for quality and productivity. 

This example uses the NASA/SEL [McGa85, Bas85b] project data base. Thirteen newly 
developed components for a particular project were selected. Size range of non-comment source 
statements is from 60 to 299 LOC. Graphs for project management were made using Microsoft 
Excel. 

In step 1, "System/Project Form" is filled out. This clarify both the software functional 
requirements and the development environment. The profile of the system and the environment 
are defined. 

In step 2, the project goals are determined based on the system/project characteristics from 
step 1 and the managerial strategy; e.g. cost, quality level to be achieved, methodologies to be 
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employed, etc. Each goal is extended to questions and metrics by means of the GQM template. 
The ‘Quality Target" and "Managerial Metrics" are determined at this step. Cost and quality 
improvement and better usage of various methods were chosen as goals. To support manage- 
ment, the "Graph Selection Form" is provided. Six graphs were selected; those are for work rate, 
effort variance, error variance, change rate, effort distribution and test efficiency. Questions to 
achieve these goals are shown- in Chap. 2. 

In step 3, the best way to achieve GQM is discussed and appropriate activities are selected. 
These depend on the pieces of information from previous steps. Development methodologies and 
quality checkpoints are listed on a specific form. This form is used as a checklist during develop- 
ment. 

In step 4, the development process is monitored and managerial data are collected periodi- 
cally. To make the project status visible, display graphs are very helpful. The graphs used were 
selected in step 2. 

The following table shows the results of statistical analysis on the NASA/SEL project. Six 
criteria on three categories are chosen. Regression analysis was executed for the "Error Vari- 
ance- data. Analysis of variance was executed for the rest of data. Based on the graphs and this 
table, the project s current status can be found. Comments for four of 13 components are 
described below. Figure 2 shows some sample graphs. 


Rules for interpreting the results 

For each metric, there exists pattern to interpret the results. Consider the following exam- 
ples. 

( Cost ] 

(1) Work Rate: Development speed measured by NCSS per man-hour 

- In case of a low value, there are several potential problems 

* low quality 

* insufficient development environment 

* loose process control 
etc. 

(2) Effort Variance: actual effort vs. estimated effort 

- Evaluate the goodness by variation between estimated and 
actual effort 

* in the case that the actual effort is lower, the work 
rate is high (or functions could be simple) 

* in the case that actual effort is high, the interpretation 
of the results are the same as for Work Rate. 

[ Quality ] 

(l) Error Variation: the number of actual errors compared 
with the estimated (a measure of complexity) 
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Component 

number 

COST 

QUALITY 

METHOD 

Work 

Effort 

Error 

Change 

Effort Distri- 

Test Effi- 


Rate 

Variance 

Variance 

Rate 

D 

bution 
C T 

ciency 

c29 




0 

X 

x XX 

XX 

c61 

0 







c6 


X 




x XX 


c5 

X 

XX 

00 


X 

X 

0 

c46 



XX 





c7 



XX 

XX 

XX 

X 

00 

c9 







XX 

c4 



0 





c49 




XX 


X 


c43 

00 

0 

00 

0 


X 

OO 

ell 


X 

XX 



X 

XX 

c63 



00 



X 

XX 

c50 







0 


D design phase 

C code phase 

T test phase 


Assessment Criteria (except Cost Distribution) 

00 

excellent 

( >= AVE +2 O' ) 

0 : 

good 

( >= +1 O' ) 

X : 

poor 

( <= -1 O' ) 

XX 

bad 

( <= -2 O' ) 

Assessment Criteria for Cost Distribution 

XX 

very high rate 

( >= AVE +2 O' ) 

X : 

high rate 

( >= +1 O' ) 

x : 

low rate 

(<= -10') 

xx : 

very low rate 

( <= -2 O' ) 


Table 2. Component Assessment Table. 

- It assumes that the greater the complexity, the greater the 
number of errors. 

* Quality is high if the number of errors is low in 
comparison with the estimated number based on complexity. 

(2) Change Rate: the normalized magnitude of the number of 
specification changes and error modifications 

- In case that the number of specification change is large, 
there is a problem in the development methods 

* insufficient review 

* less communication with user 

* loose configuration control 

- In case that the number of error modification is large, 
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quality is considered to be low. 

- In both cases, degradation of the system can be assumed 
due to entropy because of change 

[ Methodology ] 

(1) Effort distribution: effort ratio of each phase 

- Evaluate the percentage of effort in each phase (design / 
coding / test). 

* Is the effort in the design phase sufficient? 

In the case of insufficient effort, the degree of 
specification completion is considered to be low. 

* Does it cost too much in coding phase? 

In case of too much effort, it is assumed that the 
specification is insufficient or the development 
environment is not so good. 

* Is the effort appropriate in test phase? 

In case of too little effort, it is assumed that testing 
was insufficient and the system was delivered with errors. 
In case of too much effort, it is assumed that the test 
method is not efficient and/or the quality is low so 
the test phase lasted too long. 

(2) Test efficiency : percent of machine time in the test phase 

- The ratio is high if the preparation of test cases is sufficient. 

- The ratio is high if quality is high so error modification 
effort is small. 


Assumed activities in the test phase 


Preparation 1 Machine Test [ fixed 


Error modification 


The pattern for interpretating of results can be made by combining the above heuristics. 

Comments 
c5 : 

Quality is good, but cost is high. Because of the high cost 
in design and code phases, product quality must be high. 

Some changes may have caused the rise of both design and 
code cost rate. 

(good) 

Quality is high. 

(to be improved) 

Work rate is low. 

(diagnosis) 

It is necessary to monitor the early process to avoid 
the slide of schedule. Quick feedback and effective 
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reviews axe necessary. 


c7 : 

There is a problem in the methods for development and management. 

The number of changes is large. This caused the rate of 
design and code to be too high. Because of insufficient test 
instead of high test efficiency, number of errors is also large. 

(to be improved) 

- Though test efficiency is very high, preparation, 
interpretation and error correction must be insufficient, 
because there are still many errors. 

- There are many more changes than those of other components. 

(diagnosis) 

Design or review methodology must be improved. Be 
more careful in test phase. Try to find out the potential 
errors based on the test results. More experience and 
knowledge are required to do so. 

c43 : 

It’s a very good component. The only concern b the percentage 
effort of the code phase. It is true that the difficulty 
of this component is low, but both quality and productivity 
are high. 

ell : 

There b a problem in methods for development and management. 

(to be improved) 

Because of poor design, the code phase costs too much 
and there are many errors. 

(diagnosis) 

It is necessary to improve design phase to be able to make 
a better quality document. The test method should also be 
reconsidered. 

In total, the difference between the goals and results can be evaluated from Table 1. 

From the view of COST, only component 5 was well above the standard cost. This com- 
ponent, however, achieved a high quality rating, so its project goal can be considered as achieved. 

From the view of QUALITY, four of thirteen components (7,11,46,49) did not realize their 
quality target. Error analysb indicates that most errors can be reduced by avoiding careless mis- 
takes. Component 7 has an extra problem. An unusually high number of changes extended the 
design phase and caused many errors. Further investigative action should be taken into the 
causes of those changes and the manager should be encouraged to minimize change. The quality 
target has not been achieved for these four components. 

From the view of USAGE OF METHODS, two components (6, 29) had too high a cost in test 
phase. They rated satisfactory for cost and quality however. Four component (9, 11, 29, 63) 
rated poorly with respect to test efficiency. One component (29) did not meet target in both 
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categories. There are several problems to be solved in test phase. 

All three goals can not always be achieved sufficiently. However, avoiding careless mistakes 
and improving the test method should produce a better product. 

In step 5, corrective action is taken based on the collected data and managerial graphs from 
step 4. For unachieved items in Figs. 4 and 6, the cause of each problem is pursued and an 
improvement method is discussed and executed. If something is found wrong in a certain step, 
the activities in that step are improved quickly. In this way, a project can be managed systemati- 
cally throughout the life cycle. 

The expected effects of applying SMC are quality improvement, managerial cost reduction, 
accumulation and reuse of experience and a highly visible management reporting system. 


7. Evaluation of Software Management Cycle • 

We are interested in evaluating and improving the SMC itself. Data collected at each phase 
and after release enable us to analyze the effect of the SMC. The followings are the GQM for 
evaluation of SMC. 


Goal: Evaluate the effectiveness of the SMC 
Process Conformance: 

Ql. How much managerial training was given to the manager? 
Q2. How well were the SMC methods applied? 

Domain Conformance: 

Q3. How well was the SMC procedure understood? 

Q4. How well was how to interpret graphs understood? 

Cost: 

Q5. How many hours were spent to perform SMC? 

Effect: 

Q6. What was the distribution of the management time? 

Q7. Were graphs and forms helpful for the manager? 

Feedback: 

Q8. What changes need to be made in the methodology to 
make it more effective? 

Q9. What tools or activities would make the use of SMC 
more effective? 


During development, quality/productivity metrics (set Q), methods metrics (set M) and 
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feedback metrics (set F) are necessary. Customer satisfaction metrics (set C) are required after 
release. 

Table 2 shows the classification of data. 

Four evaluation methods are provided. 

(1) How good are goals? 

Based on correlation analysis between Q, M, F and C, it can be judged that a project must be 
good if the metrics of the project include most of elements of Q, M or F which have high correla- 
tion coefficient to C. 

(2) How good are activities (methods, feedback)? 

Based on correlation analysis between M, F and C, it must be good activity if an element of 
M or F has high correlation coefficient. 

(3) How good are metrics? 

Based on regression analysis between C and Q, M, F, the metrics of a project must be good or 
predictable if the project has high regression coefficient. 

(4) How good are products? 

Based on significant test of the difference between two population (past projects’ C and 
current C), the current projects’ products must be good if the difference is statistically significant. 

The results of these analyses help to improve Software Management Cycle and update the 
knowledge of management database. 


8. Conclusion 

The concepts and use of Software Management Cycle based on the Quality Improvement 
Paradigm are described in this paper. This methodology can improve not only product quality 
but also process quality. Three concepts; goal-oriented measurement, process control and visual 
management, are important to manage a project effectively, quantitatively and objectively. 

Further plans for the SMC include: 


(1) its application to a variety of projects, analyzing the processes and accumulating knowledge 
for different project classes, and 

(2) the development of a full management support tool which covers the whole process. 

The authors are convinced that this methodology contributes to the building of an 
appropriate software engineering process for improving both quality and productivity. 
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ABSTRACT 

Reuse of products, processes and knowledge will be the key to enable the 
software industry to achieve the dramatic improvement in productivity and quality re- 
quired to satisfy the anticipated growing demands. Although experience shows' that 
certain kinds of reuse can be successful, general success has been elusive. A software 
life-cycle technology which allows broad and extensive reuse could provide the means 
to achieving the desired order-of-magnitude improvements. This paper motivates and 
outlines the scope of a comprehensive framework for understanding, planning, evaluat- 
ing and motivating reuse practices and the necessary research activities. As a first step 
towards such a framework, a reuse-enabling software evolution environment model is 
introduced which provides a basis for the effective recording of experience, the gen- 
eralization and tailoring of experience, the formalization of experience, and the (re-)use 
of experience. 
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1. INTRODUCTION 

The existing gap between the demand and our ability to produce high quality software 
cost-effectively calls for improved software life-cycle technology A reuse-enabling software lift- 
cycle technology is expected to contribute significantly to higher quality ami productivity Qual- 
ity can be expected to improve by reusing proven experience in the form of products. pro< » 
and knowledge. Productivity can be expected to increase by using existing experience rather Mian 
developing it from scratch whenever needed. 

Reusing existing experience is the key to progress in any area. Without reuse evervMung 
must be re-learned and re-created; progress in an economical fashion is unlikely During Mi*' 
evolution of software, we routinely reuse experience m the form of existing products z 
Ada components, design documents, mathematical subroutines), processes (e.g.. design ir^p-er j M ri- 
methods, compiler tools), and domam-specihc knowledge (e g,, cost models, lessons learne-i. n — 
urement data). Most reuse occurs implicitly in an ad-hoc fashion rather than as the result -,f 
explicit planning and support While reuse is less institutionalized in software engineering Mian m 
other engineering disciplines, there exist some successful cases of reuse i e product reuse Rhi.m> >\\ 
sol t ware engineering lias been successful whenever the reuse<i experience is «;elf -describing. <• z 
mathematical .subroutines, or the stability o| the context in which ehe experience us reused uu- 
pensates for the lack of self-description, e g., reuse of high-level designs across projects with -mil- 
iar characteristics regarding the application domain, the design methods, and the personnel. In 
software engineering, the potential productivity pay off from reuse can he quite high -incr u i- 
inexpensive to store and reproduce software engineering experience compared to other engineer- 
ing disciplines. 

The goal of research in the area of reuse is the achievement of systematic methods for effec- 
tively reusing existing experience to maximize quality ami cost benefits. Successful reuse depends 
on the characteristics of the candidate reuse objects, the characteristics of the reuse process 
* Th* :em ““vuuiLion* \s <jsH in this paper U> rornprtse the -nun* .oft war** lif<* •■>"*1'* ( lev ^Iopm<*nt in. I urutitp naa--- 
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itself, and die technical and managerial environment in which reuse takes place. Interest m 
reusability has re-emerged during the last couple of years 4. 9, 11, 12, 13, 14, 15. 16. 17. 19, 
20, 21 ; p due in part to the stimulus provided by Ada and in part to our increased understanding 
of the relation between software processes and products. 

Our increased understanding tells us that in order to improve quality and productivity via 
reuse we need a framework which allows (a) the reuse of all kinds of software engineering ■ *x t>** r 
ence. i.e., products, processes and knowledge, (b) the better understanding of rhe reuse proc-ss 
itself, and (c) the better understanding of the technical and managerial evolution environment m 
which reuse is expected to be enabled. 

This paper presents a reuse-enabling software evolution environment model. ■ tu* hr-t -mp 
towards a comprehensive framework for understanding, planning, evaluating and motivating 
reuse practices and the necessary research activities Section 2 motivates the necessary '■cop** oi i 
comprehensive reuse framework and the important role of a reuse-enabling software evolution 
environment model within such a framework. Section 3 introduces the reuse-enabling software 
evolution environment inode! and discusses its ability to explicitly model the recording of experi- 
ence. the generalization and tailoring of experience, the formalization of experience, ami im* ■ r 1 - 

use of experience. The TAME model, a specific instantiation of the reuse-enabling 

lution environment model, is presented in Section 4 This specific instantiation is used T o aiur- 
specificaliy describe the integration of the recording and (re-)use activities into an i mpr< .vmmm ; 
oriented software evolution process. 

Before we proceed, we deiine some crucial terms that will be used in this paper so rhe reader 
understands what we mean by them in the software context. We have tailored Webster's general 
definitions of these terms to the specific domain of software evolution, Improvement means 
enhancing a software process or product with respect to quality and productivity L taming re- 
activity of acquiring experience by instruction (e.g., construction) or study (e g., analysis) lieu*? 
is the activity of repeatedly using existing experience, after reclaiming it, wuh nr withmii 
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modification. Feedback means returning to the entry point of some process armed with the 
experience created during prior executions of the process. We use the expression experience base 
to mean a repository containing all kinds of experience. An experience base can he implemented 
in a variety of ways depending on the type of experience stored. An experience base mav consist 
of one or more of the following; traditional databases containing factual pieces of information, 
iniormation bases containing structured information, and knowledge bases including mechanisms 
for deducing new information '5, 24 

2. SCOPE OF A COMPREHENSP/E REUSE FRAMEWORK 

Reuse in most environments ts implicit and ad-hoc When it is explicit (J r planned, it 
predominantly deals with the reuse of code. In Section l. we expressed our belief t.liat effective 
reuse technology needs to be based on (a) the reuse of products, processes and knowledge ib) a 
good understanding of the reuse process itself, and (c) a good understanding of the reuse -enabling 
software evolution environment. 

To better justify these beliefs, we will describe and discuss the reuse practice ;n rfir 
Software Engineering Laboratory (SEL) at NASA Goddard Space Flight ( enter 2. I s Phis m 

an example where reuse has been quite successful at a variety of level*. albeit predominantly 
implicit. Ground support software for satellites has been developed for a number of vmars m 
FORTRAN. Reused experience exists in the people, methods, ami rnols .as well m the program 
library and measurement database. 

To explain reuse in this environment we must first explain the management structure. 
There are two levels of management involved in the technical project management The second 
level managers (one from NASA and one from Computer Sciences C orporation, the contractor j , 
have been managing this class of projects for several years. Specific project managers are tvpi- 
'"dly promoted from within the ranks, on either side, from the b*-t.r »t developers on prior projecis. 


5642 


3-26 



This provides a continual learning experience for the management team. Technical review and 
discussion is informal but commonplace. Lessons learned from experience are used to improve 
management’s ability to monitor and control project developments. 

The organizational structure has been relatively constant from project to project Then' 
have been minor variations due to improvements in such things as methods and tools which have 
evolved from experience or been motivated the literature and verified by experimental -fata 
analysis on prior projects. 

The basic systems have been relatively constant. This permits reuse of the application 
knowledge as well as the requirements, and design. For example the requirements documents in* 
quite mixed with regard to the level of specificity. In some places they are quite pr**-’i>e dm m 
other cases the are very incomplete, relying on the experience of the people from prior promts 

Requirements documents have phrases similar to the following: Capability X for new satel- 
lite S2 is similar to capability X for satellite Si except for the following... This implicitly pro- 
vides reuse of prior requirements documents .as well as implicitly allows for reuse of prior design 
documents and code. 

Systems within a class, all have a similar design at the top level and the interfaces among 
subsystems are relatively well defined and tend to be relatively error free. Design is implicitly 
reused from system to system as specified by the experienced high level managers 

Reuse at the code level is more explicit. The software development process used is a reuse 
oriented version of the waterfall model The coding phase begins by seeding the code library with 
the appropriately specified elements from the appropriate prior projects. These code components 
are then examined for their ability to be reused. Some are used as is. others modified minimally, 
others modified extensively, and yet others are eliminated and judged easier to develop from 
scratch. This is a reuse approach that has evolved over time and has been quite effective. 

A variety of tools have evolved that are quite application specific These include everything 
from tools that, generate displays needed lor testing to application specific system utilities 
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Knowledge about these tools has been disseminated by guidance from more senior members of the 
development team. 

The SEL environment is a good example of strong reuse at a variety of levels, in a variety 
of ways as part of the software development process. There has been a pattern of [earning and 
reusing knowledge, processes and products. The use of the measurement database h:is 
with project control and schedule as well as quality assessment and productivity 2. I * 

NASA is now considering changing to Ada. Several Ada projects have already been . r>m- 
pleted. This has involved an obvious loss in the reuse heritage at the code level, a.s was antici- 
pated. But it has also involved a less obvious and unexpected loss of reuse at the requirement* 
and design level, in the organizational structure, and even in the application knowledge .tp-a. 

The initial impact ol Ada was staggering because of the implicit, rather rnan -xnii u 
understanding of reuse in the environment. This understanding of reuse nerds to be formalized. 

Based upon the concept that reuse is more than just reuse of code and that it ne^is t<, \„> 
explicitly modeled, we need to reconsider how we measure progress in reuse Tin- meaMinum-nr *. 
currently used in rhe SEL are based upon lines of code reused from one project to another <uv-n 
this view, progress may not b»* relaied at all to the lines of code reused WV n *rd *o ui.-.-lmip* • 
effects ol reuse on the resources expended in the entire software life cycle and on the pialitv of 
the products produced using an explicit reuse oriented evolution model. In fart. rim pro,'--* 
should allow us measure for any set of reuse -related goals 3, I. S, 10 Changing our model* and 
our metrics will help us to better understand the effects of the traditional reuse prartir*>> uni 
compare them with the effects of an explicit reuse oriented reuse model. 

In summary, we believe that a comprehensive reuse framework needs to include [a} a reuse- 
enabling software evolution environment model, (b) detailed models of reuse and learning, and (<■( 
characterization schemes for reuse and learning based upon these models. 
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3. A REUSE-ENABLING ENVIRONMENT MODEL 

In the past, reuse has been discussed independent of the software evolution environment 
We believe reuse can only be an effective mechanism if it is viewed as an integral part 
paired with learning, of a reuse-enabling software evolution environment. None of *ru- 
traditional engineering disciplines has ever introduced the reuse of building blocks as indepen- 
dent of the respective building process. For example, in civil engineering people nave not 
created "reuse libraries" containing building blocks of all shapes and structures, and then tried 
to use them to build bridges, town houses, high-rises and cottages Instead, they devised a 
standard technology for huilding ‘‘erf. am types >>f buildings le g., town houses ,i through i -.<»ng pro- 
cess of understanding and learning This allowed them to define the needs for -ert.un standard 
building blocks at well-defined stages of their construction process. In the software ar>*na w»* 
have not followed this approach. * 

[f we accept the premise that effective reuse requires a good understanding of the environ- 
ment in which it is expected to take place, then we must model reuse in the context of a reuse 
enabling software evolution environment Such a context will allow us to learn how tu resist 
better. The iiltimate expectation is that such improvement would [end T <> an -‘vr increasing 
usage of generator-technology during software evolution The ability to automate the generation 
of products from other products reflects the ultimate degree of understanding the underlying cnti- 
struction processes. Automated processes are easy to reuse for example, in building compiler 
front -ends, we rarely reuse components of other compilers; instead, we reuse the compiler genera- 
tors which automate the entire process of building compiler front-ends from formal language 
specifications 

In Section 3 1 we discuss how learning and reuse implicitly occur m the context of tradi- 
tional software evolution environments. In Section 3.2, we discuss how learning and reuse -*an be 
explicitly modeled in the context of a reuse-enabling software evolution environment 
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3.1. Implicit Learning and Reuse 

During a workshop on "Requirements for Software Development Environments" 
held at the University of Maryland in 1085, a view of a software evolution environment w;us 
proposed that consisted of an intormation system and three information producers and '-on>u- 
niers. people, methods, and tools -*d l he iniormation system is defined hv a software evoiu- 
rton process model describing the information, the communication among people, methods 
and tools, and the activity sequences for developing and maintaining software. 

The traditional software evolution environment model in Figure 1 is a refinement of this 
earlier model 


people 
x 


methods 


tools 



v 

- products 

- management plans 

- schedules 

- project data 


PROJECT DATABASE 


Figure 1: Traditional (non— reuse oriented) Software Evolution Environment Vtodel 
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The purpose of the software evolution process is to produce output products, e.g.. design 
documents, code, from input products, e g., requirement documents. People execute this process 
manually or by utilizing available methods and tools. These methods and tools can be under the 
control of a project database. All or part of the information produced during rhis process is 
stored in a project database, e.g., products, plans such as management plans or -elmdules. pro- 
ject data. 

Typically, support for such a traditional software evolution environment model includes a 
project database and means for the interaction of people with methods, tools, and the project 
database during software evolution. The experience of people. as well as >otne of die methods 
and tools, is usually not controlled by the project database As a consequence, mis experience i* 
not owned by the organization (via the project database) but rather owned by individual 
human beings and lost entirely after the project has been completed. 

Although the ideas of learning and reuse are not explicitly reliected in the traditional 
software evolution environment model, they do exist implicitly. The experience oi the people 
involved in the software evolution process and the experience encoded in methods and tools is 
reused. In manv cases, previously developed products are reused as input products In the '.am*' 
way. products developed during one activity of the evolution process can tie r ,, u>*‘d in subse- 
quent activities of this same process. People learn (gam experience) from performing r he activi- 
ties of the evolution process. Another form of implicit, learning occurs whenever products, plans, 
or project data are stored in the project database. 

The basic problem in this traditional environment model is not that learning and reuse 
can not occur, but that learning and reuse are not explicitly supported and only because of indi- 
vidual efforts or by accident. 
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3.2. Explicit Modeling of Learning and Reuse 

Systematic improvement of software evolution practices requires a reuse-enabling environ- 
ment model which explicitly models learning, reuse and feedback activities, and integrates, them 
into the software evolution process. Figure 2 depicts such a reuse-enabling environment model 



Figure 2: Reuse-Enabling Software Evolution Environment Model 

All the potentially reusable experience, including software evolution methods and tools are 
under the control of an experience base. Improvement is based on the feedback of existing experi- 
ence (labeled with "FB" for reuse in Figure 2). Feedback requires learning and reuse Systematic 
learning requires support for the recording of experience (labeled with " R" for nvording m Figure 
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2), the off-line generalizing or tailoring of experience (labeled with "G" and "T" for generaliz- 
ing and tailoring in Figure 2), and the formalizing of experience (labeled with rt F" for formalizing 
in Figure 2). Off-line generalization is concerned with movement of experience from project 
specific to domain-specific and general; off-line tailoring is concerned with movement of experi- 
ence from general to domain-specific and project -specific Off-line formalization is concerni-d 
with movement of experience from informal to schematized and productized Systematic pmim* 

requires support for (re-)usmg existing experience (labeled with "L " for use in Figur- 2j. mu 

* 

on-line generalizing or tailoring of candidate experience (not explicitly reflected m F isur* 2. 
because it is assumed to be an integral part of the (re-)use activity). 

Although reuse and learning are possible m both the reuse-enabling and tip* r radii emai 
environment models, there are significant -differences in the way experience is vmwr-t and him 
learning and reuse are explicitly integrated and supported The basic dif|Vr»‘nc** t-n*rwc*-ri ' fs* 1 
reuse-enabling model and the traditional model is that learning and reuse become “xplhutu 
modeled and are desired characteristics of software evolution 

3.2.1. Recording Experience 

The objective of recording experience is to create a repository of well specilieii an i Nrcan- 
ized experience. This requires a precise description of the experience ro be recorded tin- dt-un 
and implementation of a comprehensive experience b;use. and effective mechanisms for -oi l»*" f mg. 
validating, storing and retrieving experience. We replace the project database >f tin- r radii ioiku 
environment model by an the more comprehensive concept of an experience base which is 
intended to capture the entire body of experience recorded during the planning and execution of 
all software projects within an organization. All information flows between the software evolu- 
tion process and the experience base reflecting the recording of experience are labeled with "I?" m 
Figure 2. 

" The attributes •on-line* an»i “off -fin*" in-iicate whet her the •*orr*sp*>n<iinn activities are perform*'! as : art or . n ■ i -•* i ^ -i - 
dent of any particular software evolution project. 
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Examples of recording experience include such activities as (a) storing of appropriately 
documented, catalogued and categorized code components from prior systems in a product 
library, (b) cataloguing of a set of lessons learned sn applying a new lechnoiogv in a knowledge 
base, or (c) capturing of measurement data related to the cost of developing a system in a meas- 
urement database. 

In the SEL example of Section 2. code from prior systems is available to the program 
library of the current project although no code object repository has been developed Me.asure- 
ment data characterizing a broad number of project aspects such as the project environment, 
methods and tools used, defects encountered, and resources spent are explicitly stored tri fin* -EL 
measurement database 2, y IS Requirements and design documents as we|j ,t> l^>-on- 
about the technical and managerial implications of various methods and tools ar- impiiyy’. 
stored in humans or on paper 

Today it is possible, but not common, to find product libraries. It is even less ''ommon to 
record process-related experience such as process plans or data which characterize the impact of 

certain methods and tools within an organization There exist two main reasons why we | 

record more process-related experience: (a) it is generally hard to modif\ “Xisung pn yurr- 
efficient!) without any knowledge regarding the processes according to which 'hey wt*r*‘ .-pcto-ii 
and (b) the effective reuse of process -related experience such as process plans or data .-ohm pro- 
vide significantly more leverage for improvement than just the reuse of products. 

3.2.2. Generalizing Sc Tailoring Existing Experience Prior to its Potential Reuse 

The objective of generalizing existing experience prior to its reuse is to make a candidate 
reuse object useful in a larger set of potential target applications. The objective of tailoring exist- 
ing experience prior to its potential reuse is to fine-tune a candidate reuse object to tit a yeritir 
task or exhibit special attributes, such as size or performance. These activities require i well 
documented cataloged and categorized set of reuse objects, mechanisms that Mipport th»- 
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modification process, and an understanding of the potential target applications Oeneraiization 
and tailoring are specifically concerned with movement across the boundaries of the "generality" 
dimension: from general to domam-spectlic and project-specific and vice versa Objectives and 
characteristics are different from project to project, and even more so from environment to 
environment. We cannot reuse past experience without modifying it to the* needs of the current 
project. The stability of the environment in which reuse takes place, as well as the origination >f 
the experience, determine the amount of tailoring required. 

Examples of generalizing and tailoring experience include such activities as (a) developing a 
generic package from a specific package, (b) instantiating a generic package for a specific type. 1 j 
generalizing lessons learned from a specific design technology for a specific application to any 
design for that application or any application, (d) or parameterizing .1 cost model for a ^pecim* 
environment. 

In the SEL. requirements and design documents have implicitly evolved to be applicable to 
all FORTRAN projects in the ground support software domain Measurement data have b^m 
explicitly generalized into domain-specific baselines regarding defects and resource expenditure 
“J. N IS Requirements and designs are implicitly tailored towards r he needs of a new pmp-.-r 
based on the managers experience, and code is explicitly hand- modified ro rhe needs of a m-w 
project. 

In general, recorded experience is project specific. In order ro reuse tins experience m :s 
future project within the same application domain, we have to ia| generalize the recorded project 
specific experience into domain specific or general experience and (b) then tailor it again to the 
specific characteristics of the new project. We distinguish between off-line and on-line generaliz- 
ing and tailoring activities: 

• Off-line generalizing and tailoring is concerned with increasing the reuse potential of exist- 
ing process and product-related experience before knowing the precise reuse context fi.e.. the 
project within which the experience is being reused). Off-line generalization and tailoring is 
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concerned with movement across the boundaries of the specificity dimension within the experi- 
ence base: from general to domain-specific and then to project-specific, and visa versa. These 
activities are labeled with "G M and H T M in Figure 2’. An example of off-line generalization is 
the construction ot baselines. The idea is to use project-specific measurement data iV g . fault, 
profiles across development phases) of several projects within some application domain and m 
create the application-domain specific fault profile baseline. Each new project within th«* -ame 
application domain might reuse this baseline in order to control its development process a.s far 
as faults are concerned. An example of off-line tailoring is the adaptation of a general 
scientific paradigm such as "divide and conquer" to the software engineering domain 

• On-line tailoring and generalizing is concerned with tailoring candidate pro. •**>... t[l .; 

product-related experience to the specific needs and characteristics of a project and :n«* 
software evolution environment These activities are not explicitly reflected in Figure J beeau.-e 
they are integral part of the (re-)use activity An example of on-line tailoring is tin* adapta- 
tion of a design inspection method to better detect the fault types anticipated m Uie uirr-at 
project fi An example of on-line generalization is the inclusion of project specific effort data 
Irom a past project into the* domain specific effort baseline in order to better plan rhe require - 1 
resources for the current project Obviously, this kind of generalization couki ha\- been >.>-»•- 
formed off-line too. 

It is important to find a cost-effective balance between off-line and on-line 'adoring .mu 
generalization It can be expected that generalization is predominantly performed off-line, tailor- 
ing on-line. 

A good developer is capable of informally tailoring general and domain specific experience 
to the specific needs of his or her project. Performing these transformations on existing experi- 
►mcp assumes the ability to generalize experience to a broader context than the one studied, 
or to tailor experience to a specific project The letter this experience is packaged, the hotter 
°ur understanding of the environment Maintaining a body of experience acquired during a 
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number of projects is one of the prerequisites for learning and feedback across projects. 

A misunderstanding of the importance of tailoring exists in many organizations These 
organizations have specific development guidebooks which are of limited value because they “are 
written for some ideal project" which “has nothing in common with the current project and 
therefore, do not apply" 23 All guidebooks (including standards such as DOD->TI) -2 1 b? i ap* 
general and need to be tailored to each project in order to be effective. 

3.2.3. Formalizing Existing Experience Prior to its Potential Reuse 

The objective of formalizing existing experience prior to its potential reus** is to increase ^h*' 
reuse potential of a candidate reuse object by encoding it in more precise, better under-rood way-. 
This requires models of the various reuse objects, notations for making the models mop- precis*- 
notations for abstracting reuse object characteristics, mechanisms for validating these models, m i 
mechanisms for interpreting models in the appropriate context. Formalization activities are con- 
cerned with movement across the boundaries of the formality dimension within the experience 
base: from informal to schematized and then to productized. These activities are labeled with 
"F" in Figure 2. 

Examples of formalizing experience include such activities as (a} writing t unct >onai 
specifications for a code module, (b) turning a lessons learned document into a nianag ,, m* 4 fi t s *y-- 
tem that supports decision making, (e) building a cost model empirically based upon tin* data 
available, (d) developing evaluation criteria for evaluating the performance of a particular 
method, or (e) automating methods into tools. 

In the SEL, measurement data have been explicitly formalized into cost models 1 and error 
models enabling the better planning and control of software projects with regard to cost estima- 
tion and the effectiveness of fault detection and isolation methods 2, ti, >v IS bosons learned 
have been integrated into expert systems aimed at supporting the management decision process 
o. 2 4! 
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The more we can formalize? experience, the better it can be reused. Therefore, we try not 
only to record experience, but over time to formalize experience from entirely informal (e.g., con- 
cepts). to structured or schematized (e.g. : methods), or even to completely formal (e.g., tools) 
The potential for misunderstanding or misinterpretation decreases as experience is described more 
formally. To the same degree the experience can be modi lied more easily, or in the case of 
processes, it may be executed automatically (e g., tools) rather than manually (e.g., methods). 

3.2.4. (Re-) Using Existing Experience 

The objective of reusing existing experience is to maximize the effective use of previously 
recorded experience during the planning and execution ot ill projects within an organization. 
Hus requires a precise characterization of the available candidate reuse objects, a precise charac- 
terization of the reuse-enabling environment including the evolution process that is expected to 
enable reuse, and mechanisms that support the reuse of experience We must support the (re-jus^ 
of existing experience during the specification of reuse needs in order to compare them with 
descriptions of existing experience, the identification and understanding of candidate, the evalua- 
tion of candidate reuse objects, the possible tailoring of the reuse object, the integration of u \r 
reuse object into the ongoing software project, and the evaluating of the project's success All 
information flows between the experience base and the software evolution process reflecting 
(re-)use of experience are labeled with "U" in Figure 2 

Examples of reusing experience include such activities as (a) using code components from 
the repository, (b) developing a risk management plan bttsed upon the lessons learned from apply- 
ing a new technology, (c) estimating the cost of a project based on data collected from past pro- 
jects, or (d) using a development method created for a prior project. 

In the SEL. reuse needs are informally specified as part ol the requirements document. 
Matching candidate requirements and design documents are identified by managers who are 
experienced in this environment The evaluation of those candidate reuse objects is in parr based 
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on human experience and in pare on measurement data. They are tailored based on the 
application-domain knowledge of the -personnel. They are integrated into a very stable evolution 
process based on human experience. All this reuse Is implicit except for the reuse of code, which 
although explicit, is informal It could only be successful because it evolved within a very stable 
environment. The recent change from FORTRAN to Ada has resulted in drastic changes of this 
environment and as a consequence to the loss in the implicit reuse heritage. 

Since the key for improvement of products is always improvement of the process creating 
those products, we need to put equal emphasis on the reuse of product and process oriented 
experience. Even today, we have examples of reuse of process experience such as process 
plans (standards such as DOD-STD-'J U>7, management plans, schedules) or process data :, *iTor. 
effort or reliability data that deline baselines regarding software evolution processes wirhm a 
specific organization) In most of these cases the actual use of this information within a specific 
project context is not supported, it is up to the respective manager to find the needed informa- 
tion, and to make sense out of it in the context of the current project. 

4. TAME: AN INSTANTIATION OF THE REUSE-ENABLING ENVIRONMENT 
MODEL 

The objective of the reuse-enabling software evolution environment model of >'*cnon ■’> *J is 
to explicitly model the learning and reuse -related activities of recording experience, generalizing 
and tailoring experience, formalizing experience, and (re-jusmg experience so that r hey can be 
understood, evaluated, predicted and motivated. 

In order to instantiate a specific reuse-enabling environment, we need to choose a model of 
the software evolution process itself fn general, such an evolution process model needs to be capa- 
ble ot describing the integration of learning and reuse into the software evolution process In par- 
ticular. it needs to be capable of modeling when experience is created and recorded mro the 


- 17 - 


5642 


3-39 


experience base as well as when existing experience is used. It needs to provide analysis for the 
purpose of on-line feedback, evaluating the application of all reuse experience, and off-line feed- 
back for improving the experience base. 

The reuse-enabling TAME environment model depicted in Figure 3 is an instantiation of 
the reuse-enabling software environment model of Section 3.2. based wn a verv ^-nerni improve- 
ment oriented evolution process model. 



Figure 3: Reuse-Enabling "TAME” Environment Model 

Each software project performed according to this improvement oriented evolution [tron^ 
modfd consists of a planning and an *‘xecu f.ion stage. Tin* planum" >t.asre include a .•li:tra<-r«T!/a- 

- 18 - 

3-40 

5642 



OFHQiMrtL PAGE IS 

OF POOR QUALITY 


uon of the current status of the project environment, the setting of project and improvement 
goals, and the selection of construction and analysis methods and tools that promise to meet the 
stated goals in the context of the characterized environment. The execution stage includes the 
construction of output products and the analysis of these construction processes and result, ms out- 
put products. 

The TAME environment model gives us a basis for discussing the integration of tne r^’ord- 
mg and (re-)use activities into the software evolution process. During the environment character- 
ization stage of the improvement oriented process model we (re-)use knowledge about rhe needs 
and characteristics of previous projects and record the needs and characteristics of *h>‘ 'urr^nt 
project into the experience base. During the goal setting stage we (re-jus** existing piaus Mr i> 
struction and analysis from similar projects and record the new plans which have b****n taiArM r ■ 
the needs of the current project into the experience base. During the method and o oi 
stage, we (re-)use as many of the constructive and analytic methods and tools which had been 
used successfully in prior projects of similar type as feasible and record possibly tailored versions 
of these methods and tools into the experience base. During construction we apply rhe 
methods and tools, and record the constructed products into the experience base Dunns anai> -o 
we use the selected methods and tools in order to collect and validate data and analw** ;mun m ; 
record the data, analysis results and lessons learned into the experience has**. 

Tlie TAME environment explicitly supports the capturing: of all kinds of experience Tin* 
consistent application of the improvement oriented process model across all projects wirliin in 
organization provides a mechanism for evaluating the recorded experience, helping us to decide 
what and how to reuse, tailoring and analyzing. TAME supports continuous learning. The expli- 
cit and comprehensive modeling of the reuse-enabling evolution environment including the experi- 
ence base, the evolution process, and the various learning and reuse activities (>*■** f igure :{) allows 
us to measure and evaluate all relevant aspects of reuse. The measurement methodology used and 
supported within the TAME environment has been published in earlier papers 7. A 
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5. CONCLUSIONS 

In this paper we have motivated and outlined the scope of a comprehensive reuse frame- 
work. introduced a reuse-enabling software environment model as a first step towards such a 
comprehensive reuse framework, and presented a first instantiation of Mich an environment m tin- 
context of the TAME (Tailoring A Measurement Environment) project at the University of Mary- 
land 7. 3j. 

The reuse-enabling software evolution environment model presented in Section 3 provides a 
basic environment for supporting the recording of experience, the off-line generalization md 
tailoring of experience, the off-line formalization of experience, and rhe ( p» j »f - viom z 

experience. 

Further steps required towards the outlined reuse framework are more specific jn^d'd- ,f 
each of these activities that differentiate the components of these activities and serve as a basis 
for characterization, discussion and analysis We are currently taking the reuse-enabling software 
environment model of section 3.2 down one level and developing a model for (re-)wsmg experi- 
ence. Based on this reuse model we will develop a reuse taxonomy allowing for f h o -marnM.-nza- 
tion of any instance of reuse The reuse model will provide insight into rhe other a.-tivin— •-(' - 
reuse-enabling environment model only in rhe way they interact with the (p*.)use acrivu'. 
Corresponding models for each of the other activities need to be developed and integrated into 
the reuse-enabling software environment model. 

The reuse— enabling TAME environment model serves as a basis for better understanding, 
evaluating and motivating reuse practices and necessary research activities Performing projects 
according to the TAME environment model requires powerful automated support for dealing with 
the large amounts of experience and performing the complicated activities of recording, generaliz- 
ing tailoring, formalizing, and (rc-)usmg experience. Indispensable components of such an 
automated support system are a powerful experience b;ise. and a measurement support -yMeat. 
Many of the reuse approaches in the past have nsMtmed that the develops has Milti-nuit implim 
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knowledge of the characteristics of the particular project environment, specific needs for 
reuse, the candidate reuse objects, etc. It is 'not trivial to have all this information available. 
The institutionalized learning of an organization and the proper documentation of that 
knowledge is definitely one of the keys to effective reuse. This leads to even better specification 
methods and tools (one of the frequently mentioned keys to effective reuse) 

.\s part of the TAME project at the University of Maryland we have been working on pro- 
viding appropriate support for building such an experience base, and supporting learning and 
(re-)use via measurement. We have completed several components towards a first prototype 
TAME system. These components include the definition of project goals and their refinement into 
quantifiable questions and metrics, the collection and validation of data, their analysis ami =lu* 
storage of all kinds of experience. One of the toughest research problems is to use measurement 
not only for analysis, but also for feedback (learning and reuse) and planning purposes. W need 
more understanding of how to support feedback and planning The FAME system is intended to 
serve as a vehicle for our research towards the effective support of explicit learning and reuse as 
outlined in this paper. 
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INTRODUCTION 

The Ada programming language and the associated 
software engineering disciplines have been de- 

scribed as one of the most significant develop- 
ments in software technology in many years. 

Although many claims have been made about its ad- 
vantages and impacts, there have been very few 

empirical studies that clarify the impact of Ada. 

The Software Engineering Laboratory (SEL) is an 
organization sponsored by the National Aeronautics 
and Space Administration (NASA) consisting of 
three principal members: NASA/Goddard Space 

Flight Center, the University of Maryland, and 
Computer Sciences Corporation. The SEL was 

founded in 1976 to carry out studies and measure- 
ments related to evolving software technologies 
[7]. The studies are aimed at understanding both 
the software development process and the Impacts 
that evolving software practices may have on the 
software process and product. Since 1976, the SEL 
has conducted over 65 experiments by applying se- 
lected techniques to specific development efforts 
and measuring the resulting process and product. 

In early 1985, the SEL initiated an* effort to 
study the characteristics, applications, and im- 
pacts of Ada. Beginning with a relatively small 
practice problem (6000 source lines of Ada), the 
SEL has collected detailed development data from a 
total of eight Ada projects (some of which are 
still ongoing). The projects range in size from 
6000 lines to approximately 160,000 lines of code. 

PROJECT BACKGROUND 

Development Environment 

All Ada projects studied were developed in a 
DEC environment, using either a VAX 11/780 or a 
VAX 8600. Both machines are shared with other 
general users, and the support was average com- 
pared to other typical projects developed in 
FORTRAN. As will be pointed out later, varying 
degrees of use were made of the available tools 
and methodologies. 

COPYRIGHT 1909 BY THE ASSOCIATION FOR COMPUTING MACHINERY. INC. 
Permission to cooy without lee all or part of this matenai is granted provided 
that the copies are not made or distributed for direct commercial advantage, the 
ACM copyright notice and the title of the publication and its date appear, and 
notice is given that copying is by permission of the Association for Computing 
Machinery To copy otherwise, or to republish requires a lee and/or specific 
permission 


In studying the series of Ada projects, the 
goal/question/metric (GQM) paradigm [3] was fol- 
lowed. The goal of the study was to determine the 
impact of Ada on productivity, reliability, reuse, 
and general product characteristics. A second 
interest was to study the use of Ada features 
(such as generics and strong typing) over time. 

Project History 

Information on six Ada projects was analyzed 
for this study. The projects were developed over 
a span of 4-1/2 years starting in late 1984 and 
ending with two projects that will be completed in 
1989. The study categorized the six projects into 
three groups distinguished solely by approximate 
start date: the first two are called the first 

Ada projects, the next two are the second Ada 
projects, and the most recent are the third Ada 
projects. The timeline for the six projects is 
shown in Figure 1 . 

The first experiences with Ada in the SEL oc- 
curred with two projects that were initiated in 
late 1984 and early 1985. A team of seven pro- 
grammers was formed in late 1984 and began exten- 
sive training in Oecember of 1985. The first 
target project was a simulator that was required 
to model an attitude control system of a particu- 
lar NASA satellite, the Gamma Ray Observatory 
(GRO). Comparable simulators had been developed 
in the past by NASA, and this particular Ada proj- 
ect (GRODY) was developed in parallel to the iden- 
tical project being developed in FORTRAN. The 
results of that particular comparison are docu- 
mented by Agresti etal. [1] and McGarry and 
Nelson [14], It was estimated that the develop- 
ment of GRODY would probably require from 10 to 
12 staff-years of effort, considering previous 
experiences with similar FORTRAN projects. 

The GRODY team had an average of nearly 5 years 
of experience with software development; however, 
none had any previous Ada experience. In fact, 
there had been no earlier Ada experience by anyone 
in this environment, so no lessons learned and no 
Ada experts were available to team members. The 
team was experienced with flight dynamics problems 
although it was, on the average, less experienced 
than the typical development teams in this envi- 
ronment. 

To prepare for the design and development of 
GRODY, the team underwent 6 months of extensive 
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Figure U Ada Projects In the Flight Dynamics Division of NASA/Goddard 


training, which covered Ada syntax as well as 
principles of software engineering and detailed 
design techniques [161. The training Included the 
fol lowi ng: 

e Alsys video tapes with discussion 

• Lectures on Ada from University of 
Maryland staff 

• Lectures and workshops on PAMELA [8], 
object-oriented design, and other de- 
sign techniques 

• Workshops based on 8ooch*s training 
materials [51 

• Lectures on software engineering prin- 
ciples, Including abstraction and in- 
formation hiding 

During the 6 months of training, the electronic 
mall system CEMS) Ada project was developed. This 
was the first Ada effort completed by this organi- 
zation. It was set up as a training problem, but 
detailed statistics were kept so that the devel- 
opment process and product could be analyzed. 
When completed, EMS consisted of approximately 
6000 lines of Ada code. 

These first Ada projects, GROOY and EMS, were 
developed by the same personnel In a similar soft- 
ware development environment. The EMS project was 
developed on a VAX 11/780 using the DEC Ada Com- 
pilation System (ACS); the GROOY project was de- 
veloped on the VAX 8600. also using the DEC ACS. 

The two projects classified as the second Ada 
projects both began in early 1987 and were of 
medium size with complexity typical of other ef- 
forts In this environment. 8oth projects were 
simulators required to support the GOES mission. 
GOADA was a dynamics simulator similar to GROOY; 


GOESIM was a telemetry simulator that would be 
used In testing the attitude ground support soft- 
ware. 

The GOADA team consisted of approximately seven 
people, some of whom were not assigned full time 
to this project. Of these seven, three had pre- 
vious Ada experience and two had experience in the 
application area. The GOESIM team consisted of 
four people, one of whom had previous Ada experi- 
ence and one who had experience with this type of 
application. The training consisted of lectures 
and video tapes on Ada, with particular attention 
to the Ada style guide that had recently been de- 
veloped; lectures and classes in Ada syntax and 
Ada ^ncepts wer® also held. The training lasted 
a K ' 'ox irately i weeks for each team. 

Both of these second Ada projects used the OEC 
ACS to complete development. Because there had 
been previous Ada experience from the first Ada 
projects, experienced Ada programmers were avail- 
able to these teams for consultation and guidance. 
They proved to be heavily used commodities. The 
second Ada projects* spanned approximately 18 months 
each. 

The final two projects analyzed in this study, 
the third Ada projects, were both started In early 
1988. The UARSTELS project had requirements and 
characteristics similar to GOESIM, one of the sec- 
ond Ada projects. The other project, FDAS, was a 
much difference type of system, a source code man- 
ager used for manipulating flight dynamics soft- 
ware components. 

The UARSTELS team consisted of three people, 
one of whom had previous Ada experience; the FDAS 
team consisted of four people, none of whom had 
previous Ada experience. Training for both teams 
took the form of lectures and workshops based on 
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the Ada style giJIde; additional training from per- 
sonnel with previous Ada experience was also pro- 
vided. All team members attended classes on Ada 
syntax and Ada development. 

The teams supporting the third Ada projects had 
more personnel available to them who had previous 
Ada experience, and they also had available sev- 
eral lessons-learned reports that had been devel- 
oped by the earlier Ada projects. These two 
projects spanned approximately 15 to 18 months 
each . 

Data Collection 

Detailed data for this study were collected for 
the six projects. As for all software developed 
In the flight dynamics environment, the data are 
collected from the following sources: 

• Data collection forms 

t Tools and accounting records 

• Interviews and subjective Information 

The most extensive and detailed data were provided 
on a series of data collection forms completed by 
both the developers and managers and including the 
following: 

• Effort data (hours spent weekly by 
activity) 

• Error data (for all changes and errors) 

t Project estimation (managers' estimate 
of final size, cost, schedules) 

• Product origination (characteristics 
of modules as they are designed) 


These data were the major source of Information 
used In the comparisons. 

The tools used to record Information Include 
the automated accounting system (for records of 
computer time used, source code changes, and 

source code size history); configuration control 
tools (Configuration Management System (CMS), used 
to record changes to source code); and ASAP, the 
Ada Static Source Code Analyzer [9], which calcu- 
lates detailed counts and characteristics of Ada 
source code. Information was also provloed 
through Interviews with team developers and man- 
agers (to record lessons learned and general Im- 
pressions) and through subjective assessments made 
by senior software engineers (on topics such as 
methodologies applied). All this Information is 
consistently collected, quality assured, and re- 
corded on a data base where It Is used as the 
basis for study or analysis. 

EVOLVING CHARACTERISTICS OF THE SOFTWARE 

Design Characteristics 

The first Ada project did not automatically 
assume the reuse of the standard FORTRAN project 
methodologies and products. Ourlng the predesign 
phase, the project decided on the products, re- 
views, and methodology to be used. Project per- 
sonnel decided to conform to the traditional 
design review process that had been used In the 
flight dynamics environment for many years (2, 
15]. However, project members investigated sev- 


eral design methodologies and eventually applied a 
modified version of object-oriented design [1, 12, 
19]. 

Use of an object-oriented design required a 
rethinking of the design, Its documentation, and 
Its presentation. This caused some Inconsisten- 
cies with the traditional approach used In the 
SEL. Project members were required to develop new 
design products because the existing design docu- 
mentation methods for structured and functionally 
oriented software design did not apply to the 
object-oriented design [17], To develop a design 
notation used In the design documentation, they 
combined concepts from George Cherry's process 
abstraction method, PAMELA C8J, and Grady Booch's 
object-oriented design [5]. Design diagrams were 
presented at the preliminary design review (PDR), 
and top-level package specifications were devel- 
oped for the critical design review (CDR) . These 
components were then expanded and compiled during 
Build 0 of the implementation phase. 

The second and third Ada projects adopted and 
built on the same methodology and design nota- 
tion. They also conducted design reviews, but the 
specific products generated at each stage were 
somewhat modified from the first Ada efforts. For 
these later projects, package specifications were 
developed for the POR, and package bodies and sub- 
unit program design language (PDL) were developed 
for the CDR. In addition, the components were 
compiled before the CDR. These efforts pointed 
out the need to redefine the specific products 
produced at key milestones of the design process; 
however, the characteristics of the products have 
yet to be decided. Qulmby and Esker [17] present 
a more detailed analysis of the evolving charac- 
teristics of both the design process and the prod- 
ucts developed. 


Software Sl 2 e 

Traditionally, software size has been described 
In terms of the lines of code developed for the 
system. Lines of code can, however, be expressed 
by many measurements [11], Including the following: 

• Tota. physical lines of code (carriage 
returns) 

• Noncomraent/nonbl ank physical lines of 
code 

• Executable lines of code (ELOC) (not 
Including declarations) 

• Statements (semicolons In Ada. which 
Include declarations) 

Table 1 describes the size of the Ada projects In 
the flight dynamics environment using these four 
measurements. For comparison to the Ada projects, 
typical FORTRAN projects of similar applications 
are also summarized. 

Unless only Ada statements are counted, these 
figures indicate that using Ada results in many 
more lines of code than using FORTRAN. The In- 
crease In lines of code is not necessarily a nega- 
tive result; rather, it means that the size of the 
system Implemented in Ada will be larger than an 
equivalent system In FORTRAN. It Is also clear 
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Table 1. Software Characteristics of Projects 


APPLICATION 

1ST ADA 
PROJECTS 

2ND ADA 
PROJECTS 

3RD ADA 
PROJECTS 

FORTRAN 

PROJECTS 

DYN SIM 

DYN SIM 

TM SIM 

CONFIG 

TM SIM 

DYN SIM 

TM SIM 

TOTAL LINES 
(CARRIAGE RETURNS) 

128,000 

162,200 

87,500 

67,700 

75,000 

45,500 

28,000 

NONCOMMENT/ 

NONBLANK 

60,000 

79,900 

42,300 

36,000 

41,400 

26,000 

15,000 

EXECUTABLE LINES 
(NO DECLARATIONS) 

40,250 

49.000 

25.500 

19,700 

22,150 

22,500 

12,500 

STATEMENTS 
(SEMICOLON - INCLUDES 
DECLARATIONS) 

22,500 

29,200 

16,300 

12,700 

15,200 

22,300 

12,000 


Table 2. Effort Distribution (Percent of Total Effort) During Each 
Life-Cycle Phase of Ada and FORTRAN Projects 


PHASE 

1ST ADA 
PROJECTS 

2ND ADA 
PROJECTS 

3RD ADA 
PROJECTS 

FORTRAN 

PROJECTS 

REQUIREMENTS 

ANALYSIS 

8 

4.3 

6.7 

12.5 

DESIGN 

24 

30.5 

36.0 

22.5 

CODE 

42 

SZQ 

44.0 

35.0 

TEST 

26 

13.2 

13.3 

30 


that a precise definition Is needed of what con- 
stitutes a line of code In Ada and what types of 
code are Included In that measurement. 

Many factors contribute to the Increased size 
of the Ada projects. The style of Ada results In 
code growth because It encourages formatting, 
blank lines, and longer, readable names for data 
elements and subunits. The strong typing within 
Ada also produces more code than In FORTRAN be- 
cause each data element must be explicitly de- 
clared. In addition, the local style guide places 
further requirements on the format for readabil- 
ity. Among other requirements, the style guide 
stipulates that each calling argument must be on a 
separate physical line. All these features have 
increased the code size, but the Increased size 
also provides advancemeats In the areas of capa- 
bility, readability, and understanding. 

if.fo.rt Distribution bv Phase Dates 

Effort distributions can be described by the 
effort expended during the key life-cycle phases 
of a project and by the effort expended In soft- 
ware development activities. Using the first ap- 
proach, effort distribution by phase dates, the 
typical FORTRAN life-cycle effort distribution 
[15] in the flight dynamics environment shows 


12.5 percent of the total effort expended during 
the predesign or requirements analysis phase, 

22.5 perc2r f during the design phase, 35 percent 
during the code Implementation phase, and 30 per- 
cent during the system test phase (Table 2). 

From the review of literature on Ada [18], It 
was expected that the effort distributions would 
be significantly different for the Ada projects 
due to the modified design and Implementation ap- 
proaches. It had been anticipated that the Ada 
projects would require more effort during the de- 
tailed design phase and less effort during the 
code and test phases. However, In the flight 
dynamics environment, significant changes to the 
life cycle have not been observed. The Ada proj- 
ects were planned by managers experienced with 
FORTRAN projects, and perhaps their plans were 
Influenced by the FORTRAN life cycle. 

Although the changes are not occurring as 
quickly as anticipated, the Ada life cycle is 
changing slightly with each project and may soon 
show a different life cycle than that expected for 
a FORTRAN project. The life cycles for the second 
and third Ada projects are shifting slightly to 
show more design time required and less system 
test time. The effort distributions of the Ada 
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Table 3. Effort Distribution (Percent of Total Effort) for Devel- 
opment Activities Across All Life-Cycle Phases of Ada 
and FORTRAN Projects 


PHASE 

1ST ADA 
PROJECTS 

2ND ADA 
PROJECTS 

3RD ADA 
PROJECTS 

FORTRAN 

PROJECTS 

REQUIREMENTS 

ANALYSIS 

3.2 

3.7 

6.5 

6.0 

DESIGN 

36.5 

27.5 

35.7 

24.0 

CODE 

42.7 

52.0 

44.5 

45.0 

TEST 

17.6 

16.8 

13.3 

25.0 


projects are showing that the FORTRAN life d^cle 
cannot be automatically assumed for Ada. 

These observations probably Indicate that the 
life-cycle definition Is not easily changed merely 
because a different language or technique Is ap- 
plied. The evolution toward the expected charac- 
teristics of the new technology Is a slow, qradual 
process. 

Lffort Distribution by Activity 

The second approach to effort dl strlbutlons Is 
to analyze the effort by activity. In addition to 
collecting the effort expended between key phase 
dates (e.g., between the COR and the start of sys- 
tem test), the SEL also collects detailed effort 
data Independent of phase dates. Effort by phase 
is time driven and assumes, for example, that alt 
design activities are complete and cease at the 
end of the design phase. In reality, many activi- 
ties take place during each life-cycle phase and, 
therefore, the effort distribution by activity can 
be quite different from the distribution by phases. 

This, Indeed, was the case for the first Ada 
projects. Although only 24 percent of the total 
effort was allocated to the design phase, 
36.5 percent of the total project effort was spent 
on design activities (Tables 2 and 3). The extra 
effort needed for Ada design activities Is more 
apparent in the distribution of effort by activity 
than in the distribution by phase dates. The ef- 
fort by development activity again reinforces the 
trend seen above. On the Ada projects, more ef- 
fort was required for design and less effort for 
software testing than on the FORTRAN projects. 

Ulfi Qf Ada Features 

In an effort to achieve some measurement of the 
use of the features available In the Ada language, 
the SEL Identified six Ada features to monitor: 
generic packages, type declarations, packages, 
tasks, compilable POL, and exception handling. 
The SEL then examined the code to see how little 
or how much these features were used. The pur- 
poses of this analysis were, first, to determine 
to what degree features of Ada were used by the 
Ada project and, second, to determine whether the 
use of Ada features "matured** as an environment 
gained experience with the language. Oata on the 
use of these Ada features were obtained using the 


8 

o 
<0 
as 

3 

Ada Static Source Code Analyzer Program (9]. 
Analysis of the use of compilable POL and excep- 
tion handling did not show any trends. Perhaps it 
Is too early to see results In these areas; how- 
ever, trends were observed In the use of the other 
features. 

The average size of packages (In source lines 
of code (SIOC)) for the first Ada projects Is much 
larger than the average size of packages for the 
second and third Ada projects (Figure 2). This 
Increase is due to a difference in the structuring 
method between the first Ada projects and all sub- 
sequent Ada projects [17]. The first Ada projects 
were designed with one package at the root of each 
subsystem, which led to a heavily nested struc- 
ture. In addition, nesting of package specifi- 
cations within package bodies was used to control 
package visibility. Current Ada projects are 
using the view of subsystems described by 
Grady Booch [6, Ch. 17] as an abstract design en- 
tity whose Interface is defined by a number of 
separately compilable packages, and the only 
nested Ada packages are generic package Instantia- 
tions. 

The generic package is a major tool In the Ada 
language contributing to software reusability. 
Reports have shown the benefits of Ada reusable 
software [18] and, in the flight dynamics environ- 
ment, use of generic packages has been increasing 
from the first to the current Ada projects. More 
than one-third of the packages on current projects 
are generic packages. Although more analysis Is 
needed, this higher use of generics possibly re- 
flects both a stronger emphasis on the development 
of verbatim reusable components and an Increased 
understanding of how to use generic Ada packages 
effectively in the flight dynamics environment. 

The use of strong typing In these software sys- 
tems was measured by the number of type declara- 
tions per thousand lines of code. Although the 
measure Itself Is not Intuitively meaningful. It 
provides a method of observing trends In the use 
of Ada type declarations. In the flight dynamics 
environment, the amount of typing Is Increasing 
over time. This may Indicate that the developers 
are becoming more comfortable with the strong typ- 
ing features of Ada and are using Its capabilities 
to a fuller extent. 
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Figure 2. Use of Ada Features 


The tasking feature of Ada Is used In the 
flight dynamics environment for the dynamics simu- 
lators. Eight tasks were used for GROOY, the 
first dynamics simulator In Ada* For subsequent 
dynamics simulators of approximately the same size 
and functionality as GRODY, design personnel de- 
termined that four tasks were sufficient to Imple- 
ment the interactive capability of the system. It 
Is expected that future dynamics simulators will 
continue to use tasking, however developers are 
now using tasking more judiciously. The third Ada 
projects are telemetry simulators, which are se- 
quential systems that do not benefit from the fea- 
ture of the language, and thus do not use Ada 

tasks. 

COST/RELIABILITY/REUSE 

Product ivit y 

Discussions on Ada productivity require careful 
Interpretation because so many definitions exist 
for software size measures In Ada. Depending on 
the measurement used, software developers using 
Ada can be shown to be either as productive as or 
not as productive as software developers using 
FORTRAN. Using the total lines of delivered code 
as a measure, the Ada projects studied show an 

improving productivity over time, and they show a 
productivity greater than FORTRAN (Figure 3). 
However, considering only code statements (semi- 
colons for Ada or excluding all comments and con- 
tinued lines of code in FORTRAN), the results are 
different. An Increasing productivity trend re- 


mains in the Ada projects over time, but the Ada 
projects have not yet achieved the productivity 
level of FORTRAN projects. 

In the flight dynamics environment, many soft- 
ware components are reused on FORTRAN projects. 
8ecause no Ada components existed previously, the 
first Ada projects were. In fact, developing a 
greater percentage of their delivered code than 
the typical FORTRAN project. A past study by the 
SEL and experience with FORTRAN projects Indicated 
that reused code costs approximately 20 percent of 
the cost of new code C2]. Using this estimate, 
reusability can be factored into software size by 
estimating the amount of developed code. De- 
veloped code Is calculated as the amount of new 
code plus 20 percent of the reused code. with 
software reusability factored In, the productivity 
for developed statements on Ada projects Is ap- 
proximately the same as that for FORTRAN projects 
(Figure 4). 

Many objections are often made when computing 
productivity In terms of lines of code: It Is 

affected by style, there are many ways to code the 
same function, etc. The most Intuitive measure to 
use In computing productivity is cost per "func- 
tion." Some attempts have been made within the 
SEL to compare the functionality of projects being 
compared (e.g., GROOY versus GROSS), and data seem 
to indicate that comparing "statements* Is the 
closest measure to comparing functionality. 

The trends in Ada productivity are very posi- 
tive In that the overall cost of producing an Ada 
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Figure 4. Ada Cost/Productivity of Developed Code 


system has quickly become equivalent to that of 
producing a FORTRAN system. The flight dynamics 
FORTRAN environment Is stable, mature, and built 
on a long and extended legacy of experience. 
Although the first Ada project required 30 percent 
more staff-hours to complete than a similar 
FORTRAN project, this overhead Included effort to 
develop new practices and processes and to learn a 
new environment. With experience, the environment 
Is becoming more stable and productivity Is In- 
creasing. 

Reliability 

As with productivity, the many ways of measur- 
ing software size affect the results of reliabil- 
ity studies. For example, will the error rate 
normalized by the total system size or by the num- 
ber of language statements give the most accurate 
reading of the reliability of Ada software coie- 
pared to FORTRAN? In the flight dynamics environ- 
ment, changes to the software made after unit 
testing when the software Is placed under con- 
figuration control are formally reported on change 
report forms. The developer must supply the rea- 
son for the change (e.g., error, requirement 
change) and, If the change Is due to an error, the 
source and type of error. 


In a very mature FORTRAN development environ- 
ment, the GROSS project reported 3.4 errors per 
thousand lines of source code (KSlOC) In the sys- 
tem. As Table 4 shows, all the Ada projects 
achieved an error rate lower than the rate on the 
FORTRAN project. In addition, the error rate on 
the Ada projects shows a decreasing trend over 
time. 

When the error rate is normalized by the number 
of language statements, the first and second Ada 
projects show a slightly higher error rate than 
the FORTRAN project. However, the error rate 
again shows a decreasing trend over time. On the 
third Ada projects, the errors have decreased to a 
rate as good as. If not better than, the error 
rate on the FORTRAN project. It Is still too 
early observe a definite difference from the 
FORTRAN rates; however, the reliability of the Ada 
Projects appears at least as good as that of 
FORTRAN projects and Is improving with each Ada 
project. 

Classes of Errors 

Errors reported are classified according to 
source and type of error. Sources of errors can 
be requirements, functional specifications, de- 
sign, code, or previous changes. Types of errors 
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Table 4. Ada and Error/Change Rate 



1ST ADA 
PROJECTS 

2ND ADA 
PROJECTS' * 

3RD ADA 
PROJECTS" ■ 

FORTRAN 

PROJECTS 

ERRORS/KSLOC * 

1.8 

1.7 1.4 

1.0 

1.0 

3.4 

ERRORS/K STMTS 

10.2 

9.4 7.5 

5.3 

5.6 

6.9 


* SLOC - TOTAL LINES {INCLUDES COMMENTS/REUSED) 
- FIGURES BASED ON ESTIMATES 


are Initialization, logic/control structure. In- 
ternal Interface, external Interface, data value 
or structure, and computational. 

On a typical FORTRAN project In the flight dy- 
namics environment, design errors amount to only 
3 percent of the total errors on the project (Fig- 
ure 5). For the first and second Ada projects, 25 
to 35 percent of all errors were classified as 
design errors, a substantial increase. For the 
third Ada project, however, design errors dropped 
significantly and are estimated to be approxi- 
mately 7 percent. This rate Is close to that ex- 
perienced on FORTRAN projects and clearly shows a 
maturation process with growing expertise In Ada. 

The literature on Ada reports that the use of 
Ada should help reduce the number of Interface 
errors In the software [4]. Although the compiler 
will catch most calling parameter consistency 
errors, Interface errors can also Include errors 
that will not be detected until run time. Typi- 
cally, these are errors In string parameters or 
subtypes with different constraints and errors in 
calling parameters due to the need for additional 
or different types of parameters. Using guide- 
lines and examples In the data collection document 
[13], the errors are classified by the developer 
reporting and correcting the error. 

In the flight dynamics FORTRAN environment, 
about one-third of all errors on a project are 
Interface errors. On the first and second Ada 
projects, the percentage of Interface errors was 
not greatly reduced (Figure 5), with approximately 


one-fourth of the errors being Interface errors. 
With current projects, however, the SEL Is now 
observing a significant change: Interface errors 

are decreasing. 

In the SEL, "errors due to a previous change" 
categorizes errors caused by a previous modifica- 
tion to the software. The first Ada projects 
showed a large jump In the percentage of these 
errors compared to projects using FORTRAN (Fig- 
ure 5). However, all subsequent Ada projects show 
a rate for these errors that Is very similar to 
the FORTRAN rate. This Initial jump In the error 
rate can probably be attributed to Inexperience 
with Ada, inexperience with Ada design methodolo- 
gies. and a nested software architecture that made 
the software much more complex. Again, the error 
profile Is evolving with the maturity of the Ada 
environment. 

Software Reuse 

Throughout the years of developing similar sys- 
tems In FORTRAN In the flight dynamics environ- 
ment, the average level of software reuse has been 
between 15 and 20 percent [10, 20]. FORTRAN proj- 
ects that attained a software reuse rate of 
35 percent or higher are rare. After the first 
Ada projects and with only 5 to 6 years of matur- 
ing in the environment, Ada projects - have now 
achieved a software reuse rate of over 25 percent, 
already greater than the typical FORTRAN project. 
The UARSTELS project Is expected to consist of 
more than 35 percent reused code. This trend of 
Increasing software reuse 1$ very promising. 


DESIGN ERRORS 
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Figure 5. Error Characteristics 
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CONCLUSIONS/OBSERVATIONS/COMPARISONS 


Many aspects of software development with Ada 
have evolved as our Ada development environment 
has matured and our personnel have become more 
experienced In the use of Ada. The SEL has seen 
differences In the areas of cost, reliability, 
reuse, size, and use of Ada features. 

A first-time Ada project can be expected to 
cost about 30 percent more than an equivalent 
FORTRAN project. However, the SEL has observed 
significant Improvements over time as a develop- 
ment environment progresses to second and third 
uses of Ada. 

The reliability of Ada projects is Initially 
similar to that expected In a mature FORTRAN en- 
vironment. with time, however. Improvements can 
be expected as experience with the language In- 
creases. 

Reuse Is one of the most promising aspects of 
Ada. The proportion of reusable Ada software on 
our Ada projects exceeds the proportion of reus- 
able FORTRAN software on our FORTRAN projects. 
This result was noted fairly early In our Ada 
projects, and our experience shows an Increasing 
trend over time. 

The size of an Ada system will be larger than a 
similar system In FORTRAN when considering SLOC. 
Size measurements can be misleading because dif- 
ferent measurements reveal different results. 
Ratios of Ada to FORTRAN range from 3 to 1 for 
total physical lines to 1 to 1 for statements. 

The use of Ada features definitely evolves with 
experience. As more experience Is gained, some 
Ada features may be found to be Inappropriate for 
specific applications. However, the lessons 
learned on an earlier project play an invaluable 
part in the success of later projects. 
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USING ADA TO MAXIMIZE VERBATIM SOFTWARE REUSE 

Michael E. Stark, NASA/Goddard Space Flight Center 
Eric W. Booth, Computer Sciences Corporation 


1. INTRODUCTION 

The reuse of software holds the promise of increased productivity 
end reliability. Experience has shown that making even the 
slightest change to a "reused" piece of software can result in costly, 
unpredictable errors [Solomon, 1987]. For this reason, the Flight 
Dynamics Division (FDD) of Goddard Space Flight Center 
(GSFQ is concentrating effort on developing "verbatim" reusable 
software components with Ada, where verbatim means that no 
changes whatever are made to the component. 

This paper presents the lessons learned on several simulator 
projects in the FDD environment that exploit features of the Ada 
language, such as packages and generics, to achieve verbatim reuse. 
These simulators are divided into two separate, but related, problem 
domains. A dynamics simulator is used by the FDD mathematical 
analysts to verify the attitude control laws that a spacecraft builder 
has developed. A telemetry simulator generates test data sets for 
other mission support software. FDD began using Ada in 1985 
with the development of the Gamma Ray Observatory attitude 
dynamics simulator (GRODY). Since that time, six additional 
simulator projects have been started. With each successive project, 
a concentrated effort is made to use the lessons learned from 
previous Ada simulator development projects. 

This paper focuses on the concepts used in the projects that have 
had the most impact on verbatim software reuse in the FDD 
environment: GRODY, the Upper Atmosphere Research Satellite 
Telemetry Simulator (UARSTELS), and the Generic Dynamics and 
Telemetry Simulator (GENSIM). This paper defines underlying 
design principles, discusses how Ada features support these 
principles for reuse in the small and shows how these principles 
are used to achieve reuse in the large. Finally, this paper presents 
supporting data from current reusability results. 

The FDD has been using a modified version of the General Object- 
Oriented Development (GOOD) methodology [Seidewitz, 1986; 
Stark, 1987; Seidewitz, 1988] to develop its Ada software. Three 
concepts that play a role in GOOD enhance verbatim reuse: 
abstraction, inheritance, and problem -specific architectures. These 
concepts support the reuse of successively larger components 


within successively narrower domains. The next two sections 
describe how these concepts are applied to simulator projects in the 
FDD. Abstraction supports "reuse in the small" and inheritance 
and problem-specific architectures support "reuse in the large." The 
current practice is to support reuse in the small through component 
libraries, as is done with a collection of components by Grady 
Booch [Booch, 1987] and EVB’s Generic, Reusable Ada 
Components for Engineering (GRACE) [Berard, 1989]. The 
UARSTELS and GENSIM projects are cited to describe how reuse 
on a large scale is accomplished and to demonstrate the potential in 
cost savings and/or the ability to solve more complex problems. 


2. REUSE IN THE SMALL: 

USE OF ADA GENERICS 

This section presents design and implementation guidelines for 
using Ada generics. It shows how the design principles mentioned 
in the previous section are implemented using Ada. 

Designing individual generic components is understood to the 
point that such components are co mm erc i ally available. The Booch 
taxonomy of structures creates a family of components that satisfy 
the same abstraction within the context of different problems, for 
example, sequential versus concurrent applications. 

The design process becomes more interesting when a hierarchy of 
generic co m ponents is needed. This is the case when designing 
generic subsystems with multiple levels of abstraction. This 
leveling of a subsystem can be embodied in the Ada code by using 
generic units and instantiations in the following three ways: 

1. Library unit instantiations 

2. Nested instantiations 

3. Nested generic definitions 

This section will define each of these approaches and provide 
examples to demonstrate their application. Implications for 
reusability and lessons learned are included for each approach. 

Library Unit Instantiations 

The first approach is to create an instantiation of a generic that is a 
library unit. This approach is appealing for practical reasons. The 
potentially broad scope provided by library unit instantiations may 
be necessary for Ada compilers that do not implement code 
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sharing* {Ganapathi, 1989]. Most implementations currently do 
not support code sharing. Instead, each instantiation creates a 
complete object code copy of the generic template. The system- 
wide scope provided by library units often makes only one copy 
(instance) necessary. 

To describe the first method of library unit instantiations, assume 
that the generic package A depends on a set of subprograms, P, 
provided by the generic package B. Instantiating B as a library unit 
creates a copy of this generic package called Instance _B. The 
generic package A may then be instantiated by using the 
subprograms provided by Instance_B as actual parameters. This 
allows the generic packages A and B to be designed and 
implemented having no external dependencies, which mikes reuse 
simpler. This approach is depicted in the design diagram^ shown 
in Figure 1. 



As previously mentioned, instantiating generics as library units 
has the advantage (hat instantiations of A and B may be imported 
(named in a with clause) by any compilation unit in a system. 

When full visibility is desired, this technique works well. 
However, if 3 were an abstract state machine^ and the design 
required that B should be visible only to A, this potentially broad 
visibility of B is undesirable. It would be far better to use the 
language to enforce that design decision. 

Another drawback of library unit instantiations is that as generic 
components become more complex, they require a longer list of 
more problem- specific generic formal parameters. Each 
instantiation of the generic becomes long and complex. The 
instantiations can be made simpler by specifying defaults for 
formal subprogr am s using the notation “with procedure PI is o." 
Then (assuming in this case that As only formal parameter is PI), 
the instance of A can be written as shown in Figure 2. 


* Code sharing is a technique that allows each instance of a generic 
to share the same object code. The result is usually a smaller 
object code size and slower execution speed for the system. 

2 The notation for the design diagrams uses rounded-comer 
rectangles to represent packages, solid snows to represent 
dependencies, broken arrows to represent instantiations, and broken 
package symbols to represent a generic unit. 

3 A package that is an abstract state machine is a package that 
maintains state information in the package body [Booch, 1983], 


generic 

with procedure Pi Is o; 
package A Is 

end A; 

generic 

package B is 
procedure PI; 

endB; 

with AJnstance_B; 

use Instance_B; 

package Instance_A Is new A; 


Figure 2. 

This technique works well when numerous simple functions are 
used as formal parameters. This use of Ada simuiaics the search of 
si object code library to resolve external references. Figure 3, part 
2 presents an example of mathematical packages being used to 
instantiate the package of flight dynamics abstract data types 4 
shown in part 1. Care should be used with this technique, since 
the use clause does more than simply make objects and operations 
directly visible. There are visibility and precedence rules of the 
language that will affect what defaults will be used [Mendal, 1988]. 


generic 

type REAL Is digits o; 

type RADIANS Is digits o; 

type VECTOR is array ( INTEGER range o) of REAL; 

type MATRIX Is array ( INTEGER range o, 

INTEGER range o) of REAL; 
with function sin ( Angle : In RADIANS) 
retain REAL is o; 

with function cos ( Angle : In RADIANS) 
return REAL Is o; 
with function Floor ( Item: in REAL) 
return REAL is o; 
package Generic^Attitude.Types Is 

Figure 3 (1 of 2). 


4 A package that is an abstract data type exports objects, types, and 
operations but does not maintain state information in the body 
[Booch, 1983]. 
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with Single_Math_Functions, 

S in gle_Linc ar_Al gcbra; 
use Single_Mtth_Functions, 

Singlc_Linc*r_AIgcbra; 
with MaihJTypes, 

Generic _Amtude_Types; 

package Single_Attitude_Types Is 
new Generic _Attiaide_Types ( 

REAL => MathJTypes.SINGLE, 

RADIANS => Single_Math_Functiona.RADIANS, 
VECTOR *> S ing le_Linear_ Al gebra. VECTO R, 
MATRIX => S ingle_Linear_ A1 gebra^MATRIX ); 


Figure 3 (2 of 2). 

A second method that uses library unit instances is to instantiate B 
as a library unit, which is then wlth-ed into the body of generic A 
(Figure 4). Since the procedures do not need to be passed as actual 
parameters, this option allows A to have a shorter formal 
parameter list. However, method often requires the use of common 
types for A and B. For example, for a flight dynamics application, 
the generic package A would have to be coupled to the same 
floating-point types as the generic or instance of package B. As 
long as this sort of coupling is relatively simple, it can be 
managed (in the simulator case by having a single package 
containing the basic floating-point types). 



The disadvantage of this approach is the use of a common types 
package to implicitly couple A and B. This defeats software 
engineering principles of information hiding and data abstraction. 
The following section describes how to exploit these software 
engineering principles by nesting generic instantiations. 

Nested Instantiations 

Continuing the same example, the generic package B may be 
instantiated in the body of generic A (Figure 5) using the generic 
formats of A. This option is ideal for abstraction and information 


hiding because it can be extended to a senes of nested 
instantiations. Objects, types, and operations from B may be used 
as building blocks and specialized to raise the level of 
abstraction [Stark, 1987]. This is sometimes referred to as re- 
exported. Importing the package B in this manner allows objects, 
types, and operations to be hidden in the body of the generic 
package A but still used to instantiate B. 



Using nested instantiations is an appealing technique for software 
engineering reasons. The limited scope provided by a package 
boundary increases information hiding and protection. The amount 
of information that the user of the outer package A needs to know 
may be lmriig«! to the package specification of A. The fact that an 
instantiation of a lower level generic is used to implement the 
package is irrelevant to the user. Ramifications of future changes 
made during the maintenance phase will be much more limited in 
scope than the library unit instantiation approach. 

The disadvantage of using nesting instantiations is the advantage of 
using library unit instantiations. That is, if the Ada compiler 
being used does not support code sharing and an instance of a 
particular generic is necessary in several locations, nesting will 
result in multiple object code copies. For example, the executable 
size of GRODY is less than 2 megabytes (Mb). This simulator 
does not have multiple copies of nested instantiations. 
UARSTELS, on the other hand, makes extensive use of nested 
instantiations, which resulted in an executable size of 6 Mb. 

On the surface, the implication of these findings seems to suggest 
using library unit instantiations exclusively. The actual 
implication, however, is that one should use nested instantiations 
only when a few copies are necessary. One way to minimize the 
number of copies is to implement the generic package as an 
abstract data type rather than an abstract state machine. This 
approach may be possible when multiple instances of the 
abstraction use the same types and subprograms as actual 
parameters but use different objects (Ada allows objects to be 
passed at run time, while types and subprograms must be passed at 
instantiation time). This approach allows copies to be created with 
object declarations instead of generic instantiations; it also has the 
benefit of increased flexibility, since objects may be declared static 
at compilation time or created (dynamically) at am time. 

When the correct design calls for multiple instances of an abstract 
state machine the long-term solution is to acquire a compiler that 
supports code sharing. 
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Nested Generic Definitions 

Nested generic definitions is the third design approach described 
here. This technique is appealing when the problem calls for a 
high degree of coupling between generics. 

Changing the previous examples, if instantiations of generic 
package A and generic package B will have common types and 
subprograms as actual parameters, the following architectures are 
possible: 

1. Make the types and subprograms visible to the generic 
templates via with clauses. 

2. Make each generic package a library unit and duplicate the 
generic formal parameters in the generic pan of each. 


3. Nest the generic definitions within the specification of 
another generic package, C. The generic pan of C 
contains the common formal parameters (Figure 6). 



Although the first option works, it suffers from a high degree of 
coupling. Future instances of either A or B will always be using 
the same common types and subprograms. This inflexibility 
results in limiting the verbatim reusability. 


The second option is a large improvement over the first. Future 
users of packages A and B may now supply their own types and 
subprograms for the generic formal parameters. However, this 
architecture becomes tedious and error prone when the common 
types and subprograms are long and complex. Since generic 
instantiation becomes a large part of the effort when maximizing 
verbatim reuse, it is desirable to simplify this activity. 

The third option accomplishes the same goals as the second option 
but with less duplication. This option is useful when the number 
of nested generics or the number of common generic formal 
parameters becomes large. It is less error prone because the 
common actual parameters are supplied only once. The 
maintenance phase also benefits from the single location of 
common actual parameters. 

Figure 7 shows an example of nested generic definitions from 
UARSTELS. The generic function FSS Digitize is declared 
within the generic package Generic Sensor Digitization. The 
genenc formal parameters of the composite package (generic sensor 
digitization), as well as the generic formal parameters of FSS 
Digitize, are referenced in the body of FSS Digitize. 


generic 

type REAL Is digits <>; 
type COUNTS is range o; 
with procedure Log_ Error 

(Message : in String) is Text_IO.Put_Line; 
package Generic_Sensor_Digitization is 

function I jnear ^Digitize 
( Parameter : In REAL; 

Dias tin REAL; 

Scale: in REAL ) return COUNTS; 
generic 

type COUNTER is range <>; 
with function tan (X: REAL) return REAL Is o; 
with function sin (X: REAL) return REAL Is o; 
with function cos (X: REAL) return REAL Is o; 
with function Floor (X: REAL) return REAL is o; 
function FSS_Digitize 
( Angle : in REAL; 

Coefficient : in REAL; 

Tolerance ; in REAL; 

Maximum_Number_of iterations : In COUNTER ) 
return COUNTS; 

end Generic_Sensor_Digitiza&on; 


Figure 7. 

Finally, a practical benefit accrues from using the nested generic 
definition approach. Most compilers, as previously pointed out, do 
not support code sharing; instead, they expand the generic template 
at instantiation time. The advantage for the designer needing only 
1 generic from a package containing 10 nested generic definitions 
is that only the object code for the 1 instantiation will be 
generated. 
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3. REUSE IN THE LARGE: 

PROJECT-SCALE REUSE 

This section presents the details of the. two projects in FDD that 
have had the most impact on verbatim reuse: UARSTELS and 
GENSIM. For both projects, an overview is presented giving the 
background information, the goals, and the motivating factors 
involved during development. Each system's architecture is 
discussed using the concepts and notation from the previous 
section of this paper. Finally, the lessons learned from each 
project are discussed with their implications for future development 
efforts. 

UARSTELS Overview 

The UARSTELS project was started in February 1988 and was the 
fourth Ada simulator initiated at FDD. Previous simulators 
included the GRODY experiment, the Geostationary Operational 
Environmental Satellite-I (GOES -I) dynamics simulator 
(GO ADA), and the GOES -I telemetry simulator (GOESIM). The 
GOES-I simulators represent the first operational Ada software 
developed at FDD. 

The GRODY design team exploited the feature of nested units, 
which resulted in increased information hiding (information 
protection might be a better phrase). The rationale for using 
information hiding is increased reliability. Higher reliability 
should, in turn, increase reusability; that is, if a component is very 
reliable, it is appealing to reuse. This was the basis for the 
GRODY design. 

The lesson learned from this approach was that extensive use of 
nested packages actually decreased reusability [Quimby, 1988]. In 
addition, during the coding and testing phases, the development 
team observed the high compilation overhead incurred by the nested 
architecture. 

Given these lessons from GRODY, the GO ADA and GOESIM 
design teams developed a non-nested architecture with the twin 
goals of increasing reusability and reducing compilation overhead. 
Both these goals were met Individual software components could 
be picked up by successive projects and reused with slight 
modifications. The use of library packages, rather thin nested 
packages, kept the compilation costs to a minimum. 

One of the lessons learned from the implementation phases of the 
GOADA and GOESIM projects was that using Ada was not 
significantly decreasing the level of effort for integration testing. 
This was unexpected. It was predicted that the integration test 
phase would require less effort than past FORTRAN integration 
test phases. Some Ada developments have claimed that system 
integration took significantly less effort than for similar, previous 
non- Ada projects [Hudson, 1988]. 

Analysis by the UARSTELS design team showed that by un- 
nesting G ROD Ys packages, more objects and types became visible 
at a high level in the GOADA design. This increased the number 
of components to integrate at each level. 

UARSTELS Architecture 

The architecture of UARSTELS was influenced from the start with 
the knowledge that another, very similar simulator would follow: 
the Extreme Ultraviolet Explorer (EUVE) telemetry simulator 
(EUVETELS). A high level of reuse from UARSTELS to 


EU VET ELS was both desired and thought to be possible because 
of reused functional specifications. However, because the two 
spacecraft were themselves different, the telemetry simulators 
would be different. The design for UARSTELS needed to take into 
account these spacecraft dependencies and parameterize them. 

Each design decision made on UARSTELS attempted to satisfy the 
following requirements: 

1. All UARSTELS requirements 

2. Some known requirements from previous systems 

3. Some possible future mission requirements 

The goals of the UARSTELS team were to maximize verbatim 
reuse and allow the compiler to check system integration as much 
as possible. To achieve this, the design team took a hybrid 
approach to the system s architecture. Most packages were 
developed as library packages, rather than nested units; however, 
these packages were designed as generic units. This allowed the 
instantiations of these generic packages to be nested in successively 
higher level packages. The level of nesting (or layering) within 
UARSTELS is comparable to that of GRODY, with the important 
difference being the use of generic units in UARSTELS. The 
generic packages may be picked up and reused just as the 
nongeneric packages in GOADA or GOESIM, with the important 
difference again being the use of generics. The nested 
instantiations allow the language to perform integration checks 
(whether the programmer wants them performed or not) at each 
compilation just as in GRODY. 

A specific example of this nested instantiation approach is the 
design of each simulated sensor model within UARSTELS. In 
Figure 8, the Report. Writer, Data.Set, and Plot.File generic 
packages are library units. As such, they may be picked up and 
reused independently of each other. In the case of a sensor model, 
however, each of these objects is necessary. To provide all three 
of these abstractions to each sensor model, they are instantiated 
within the specification of the generic package Sensor.Output. 
The application-specific parameters are provided to the three low- 
level generics when they are instantiated. The sensor-specific 
parameters are generic formal parameters to Sensor.Output. 
Sensor.Output is then instantiated within the body of the generic 
sensor model package, with the sensor-specific parameters being 
provided at that time. The spacecraft-specific parameters are 
generic formal parameters of the generic sensor model package. 
The instantiation of the generic sensor model package resolves ail 
the formal parameters, resulting in the Fine_Sun_Sensor object. 

As a result of this architecture, UARSTELS differs from previous 
systems in its recompilation time and executable size. The 
increased use of g e ne ri c units had a direct effect on the compilation 
overhead. It takes longer, in CPU time and elapsed time, to 
recompile UARSTELS than any of the other simulators. In 
addition, while UARSTELS is significantly smaller in source lines 
of code and in number of components, it is also significantly larger 
in executable image size. This is because the Ada compiler used in 
FDD does not support code sharing. Instead, it expands generic 
units for each instantiation. 
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UARSTELS Lessons Learned 


The lessons learned may be summarized as follows: 

• Nesting reduces integration testing 

• Library units provide reusable software components 

• Generic library units provide reusable components and 
allow information hiding via nested instantiations 

• Many Ada implementations incur a large compilation 
time overhead for the use of nested and generic units 

• Many Ada compilers provide a simple implementation of 
generic units 

The nesting feature of Ada must be exploited. The virtue of 
nesting is information hiding (protection) and higher reliability; 
this is one of the promises of using Ada. However, like all of 
Adas promises, higher reliability does not happen automatically. 
It must be engineered. 

The strong typing feature of Ada must also be exploited. The 
definition of distinct types that are relevant to the application 
domain, such as flight dynamics, needs to be engineered. Ada can 
help eliminate the common mistakes (i.e., mixing radians and 
degrees or meters and kilometers), but this will not occur 
automatically either. Through the use of strongly typed objects, 
reliability can be further improved and integration testing can be 
further automated. 

Strong typing encourages and, in most cases, forces the use of 
nesting. Operations on private types must be defined within the 
same scope (package) that defines the type. Since the internal 
structure of that type is not visible outside this scope, all 
operations must be defined within the scope or the operations must 
be imported with a generic instantiation. 


Circumventing nesting, strong typing, and generics in order to 
minimize compilation time is a short-term fix with the long-term 
ramification of decreased reliability and reusability. If the 
compilation overhead is unacceptable, then alternative Ada 
developmait environments will be required. 

GENSIM Overview 

The GENSIM project was started in 1986 by a group studying 
ways to increase the reuse of simulator software and the possibility 
of integrating the dynamics and telemetry simulation capabilities. 
This group consisted of both software developers and mathematical 
analysts, all of whom had simulator project experience. At the 
same time, reuse studies in the Software Engineering Laboratory 
(SEL) [Solomon. 1987] showed that reusing code without 
modification (verbatim reuse) yields a tremendous reduction in 
development cost One of the early products of the GENSIM effort 
was a study that estimated that costs of software development for a 
dynamics simulator could be cut in half by creating verbatim 
reusable components. 

The GENSIM team believed that the best approach to maximizing 
verbatim reuse was to reuse products from all phases of the 
software engineering life cycle. This belief was based on developer 
experience, rather than any formal software engineering theory. 
The simulator problem was divided into "modules," each of which 
models an entity in the problem domain. The products associated 
with each module include a specification, design documentation, 
code, and test cases; each follows project-wide standards. A module 
specification consists of a complete definition of the inputs and 
outputs needed, the algorithms to be implemented to model the 
entity, and documentation of the mathematical analysis and 
assumptions underlying the algorithm. A module design is built 
according to standard protocols for initialization, computations, and 
the passing of parameters between modules. These protocols allow 
the individual modules to be configured into a standard simulator 
architecture. 

To determine the feasibility of a generic simulator, a prototype is 
being developed and applied to a simplified mission. After the 
prototype demonstrates the ability to configure a dynamics 
simulator for different missions, a full set of components and 
modules will be developed. 

GENSIM Architecture 

This subsection first describes the GENSIM dynamics simulator 
architecture, then discusses design issues for both modules and 
standard subsystems. The next subsection discusses lessons learned 
from the initial GENSIM design that can be applied to the final 
version of the system. 

Figure 9 shows the standard dynamics simulator architecture. The 
reusable modules are parts of the spacecraft, hardware, and 
environment models (SHEM) subsystem. Typical SHEM 
modules include Sun sensor modeling and geomagnetic field 
modeling. The spacecraft control subsystem is always mission 
dependent because a dynamics simulator is intended to test attitude 
control algorithms for a specific satellite. The only reusable parts 
in this subsystem are a simulated ground command uplink interface 
and a module that computes estimation and control errors. The 
user interface has the obvious capabilities of reading and editing 
input parameters and producing reports and plots from analysis 
results. The case interface subsystem is responsible for 
maintaining simulation cases, including analysis results data, 
simulation input parameters, and suspended simulation cases. The 
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case interface also standardizes communicauons between the SHEM 
modules and the user interface. The simulation executive 
subsystem has two major purposes: to manage requests to control 
the simulator and to control the sequencing and timing of module 
execution. The last subsystem is a utilities subsystem, which 
consists of several generic units and a set of standard, interrelated 
instantiations, as is described in Section 2. For example, the 
instantiation of a generic linear algebra package requires a square 
root function, which is provided by the instantiation of a general 
mathematical functions package. 



Figure 9. 


Figure 10 shows the implementation for a typical SHEM module 
(Module K), The package ModuleJC performs the actual 
modeling. It can be initialized and invoked from the simulation 
executive, and communicates with other SHEM modules through 
procedure and function calls. The generic module database and 
generic module results objects are generic packages that implement 
the standard communications between a module and the other 
subsystems. These generics are instantiated with types and values 
from the Module_K_T ypes package. These instantiations are called 
on by the Module_K package when the model itself is being 
initialized or activated, and they are called on by case interface 
components whenever parameter or results data is required by some 
other subsystem. Using this standard approach allows a different 
set of SHEM modules to be used for each mission. 



Figure 10. 


The GENSIM design just described is a conservative extension of 
existing simulator designs. In general, the dynamics simulation 
capability remains the same, but the design has been reworked to 
be more object oriented. For example, the case interface 
subsystem was added to treat the concept of simulation case as an 
object, rather than to distribute those capabilities between the user 
interface, the SHEM, and the utilities. The utilities subsystem 
was also changed from one gigantic generic package to several 
smaller, independent generic units. The main effort in GENSIM 
has been directed toward generalizing the design to make the 
components reconfigurable, rather than adding new capabilities. 

The user interface, case interface, and the simulation executive 
subsystems must be implemented as generic subsystems that can 
be parameterized by the selection of modules for a given mission. 
The case interface subsystem can be used to demonstrate how a 
generic subsystem is designed. The discussion of individual 
modules in the previous paragraph shows how the generic module 
database and module results packages are used by modules. The 
case interface subsystem must access these same instances to 
communicate with the user and to maintain simulation cases. 


Figure 11 shows the design for the generic case interface 
subsystem. The case manager object is responsible for managing 
simulation cases as a unit; and the ground command interface, 
parameter interface, and results interface objects manage 
components (such as analysis results) of a simulation case. The 
parameter interface and the results interface also mknage the 
communications with SHEM modules. Figure 1 1 shows that all 
these packages are interrelated; however, they arc used one at a 
time. For example, a procedure that edits ground commands would 
use the ground command interface but not the other objects. 
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Figure 11. 

If these objects were not being implemented as generics, each 
object in this subsystem would be implemented as a library 
package, which could be imported independently. With a generic 
subsystem, each of these packages must be parameterized, and 
many of the generic formal parameters are common to more than 
one package. If each object were implemented as a separate generic 
package, there would be multiple definitions of the same formal 
parameters with all the maintenance problems such a redundant 
structure entails. Since the packages in this subsystem are coupled 
anyway, the case interface subsystem is implemented using the 
nested generic definitions technique (described in Section 2 of this 
paper), which allows the common parameters to be placed in the 
generic part of the composite package. 

In the GENSEM design, even the parameters that apply to only one 
package were placed.in the composite package. When this is done, 
the nested packages are no longer generic. This approach reduces 
any possible confusion between generic packages and their 
instances by allowing the user to instantiate the entire subsystem. 
Then the nested packages can be used without having to instantiate 
more generics. The cost is that the nested packages are now more 
highly coupled. In the case interface, this increased coupling is 
justified because all the coupled packages are part of the abstraction 
’’simulation case.” The utilities subsystem consists of independent 
generic packages, where the coupling is introduced between 
instances of these generics. This approach allows the generic 
packages to be used outside the context of dynamics simulators. 
There is no corresponding need to use individual components of 
case interface outside the context of dynamics simulators because 


the coupling between the components is defined by the nature of a 
simulation case. The degree of coupling allowed in the design and 
implementation of reusable components is one of the key 
judgments developers must make. 



Figure 12. 


GENSIM Contributions 

The major benefits derived from the GENSIM project are in the 
categories of (I) gaining experience in the use of advanced Ada 
features, (2) getting ideas for improvements in simulator design, 
and (3) producing the reusable components themselves. This 
subsection will focus on the first two categories. In the fust area, 
the redefinition of the utilities subsystem as a set of independent 
generic packages tied together as a set of interrelated instantiations 
demonstrated the ability to write generic packages as described in 
Section 2. 

When designing a subsystem this way, the developer must make 
sure that all the generic formal parameters designed are matched by 
actual parameters provided by some other prckage. It is also eases 
development if the code is written and tested bottom up, so that the 
lower level instantiations needed to instantiate the senior-level 
generics are tested and, in turn, can be relied on to support the 
testing of other objects. If care is taken to design a set of generic 
packages with consistent naming conventions, defaults can be 
provided by standard instantiations, allowing the rapid writing of 
instantiations for testing purposes. When all these conditions are 
met, the techniques described in Section 2 work very well. The 
decoupled generics/coupled instantiations technique is the best 
approach to develop packages that provide the ability to use 
problem domain abstractions rather than predefined Ada constructs. 

GENSIM has also contributed to the use of strong typing by using 
more private types than previous simulators and by beginning to 
focus on the decision criteria for their use. The criteria for using 
private types is that they should add the protection of data integrity. 
For example, the attitude types package defines die private type 
COSINE_MATRTX. This type is identical in data definition to 


5642 


4-18 




any other 3-by-3 matrix but has a set of operations that guarantee 
that a COSINE_MATRIX always represents a rotation. In the case 
of GEN SIM ‘s orbit data types, a private type docs not add any such 
data protection; the effect is to force a user to use operations 
provided for the data type in precisely the same way as one would 
use an assignment statement. When this is the case, the type 
should be made visible in a package specification. 

Possible Improvements to GENSIM 

The GENSIM prototyping has been successful in generalizing 
dynamics simulator designs, but some features inherited from past 
simulators can be improved on. Currently, simulator module state 
data types are built from individual scalar objects or arrays of scalar 
objects. A more object-oriented design would define abstract data 
types (ADTs) for the problem domain entities and then use these 
ADTs to define module states. This is particularly true when there 
are multiple objects of a type, as is the case with spacecraft 
sensors. As an example, the current design of a fine Sun sensor 
model defines the simplified module state as follows: 


package body Fine_Sun_Sensor is 

- N is the number of Fine Sun Sensors used for a mission 
type STATE is record 
Alpha_ J Angles 

: Double_Unear_Algebnu VECTOR( 1 ..N); 

Beta _Angles 

: Double_Linear_Algebra.VECTOR(l..N); 

Alpha JLimit 

: Double_Linear_Al gebra. VECTOR ( 1 . .N) ; 

BeU_Limit 

: Double_Linear gebra. VECTOR( 1 _N); 
end record; 

Module.State : STATE; 
end Fme_Sun_Sensor ^Module; 


A better implementation is as follows: 


with Fine_Sun_Sensor_ADT ; 
package body Fme_Sun_Sensor_Module is 
type STATE is array of (1..N) 
of Fine_Sun_Sensor>DT.FINE_SUN_SENSOR; 

Module^S tate : STATE; 

end Fme_Sun_Sensor_Module; 


In the second implementation, Fine_S un_Sensor_ADT 
.FINE_SUN_SENSOR is an abstract data type that encapsulates 
all the necessary attributes for a single sensor and provides both 
selector and constructor operations. One advantage of using 
abstract data types is the tighter encapsulation of If a change 
is made to the fine Sun sensor modeling, the scope of the change 
is then restricted to the body of the abstract data type package rather 
than affecting the entire module. 

Another advantage of using abstract data types in this context is 
the strict separation of the problem domain object itself and its use 
within a software system. In the first example, the declaration of 


Alpha_Limil mixes the problem domain concept of a limited 
sensor field of view wiLh the fact that N sensors are used for a 
particular mission. The second implementation makes it clear that 
N refers to the number of sensors and that the abstract data type 
encapsulates each sensor s angles and limits. 

When the problem domain object (or class) is implemented with a 
distinct Ada library unit, it is possible to use the object-oriented 
programming concept of inheritance to create a hierarchy of classes 
and subclasses. Figure 13 shows how this could work when all 
the details of a fine Sun sensor model are considered. This 
inheritance tree, which is implemented using nested instantiations, 
shows four levels of increasing complexity, starting with the 
superclass FSS_ADT and creating a chain of subclasses from there. 
Each of these four generics can be instantiated either as a library 
unit or nested within a module. Each subclass in the chain tailors 
its superclass by incorporating the models provided by the 
respective utility packages. Inherited operations can also be 
specialized and new operations can be added to a package [Stark, 
1987]. For example, a fine Sun sensor engineering model needs 
to decalibrale simulated data so that calibration algorithms can be 
tested. Since this decalibration is specific to fine Sun sensors, the 
operation would be added to the generic FSS engineering model 
package, with the noise, biases, and misalignments being provided 
by the generic measurement utilities. 
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Figure 13. 

This approach uses the nested generic instantiations in the same 
way as UARSTELS. The difference is that all scnsor-spccific 
utilities, such as fine Sun sensor decalibraiion, arc part of the 
sensor abstract stale machine, not part of the utility packages. 
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The use of inheritance allows the selection of an appropriate model 
for a wider variety of applications. A telemetry simulator would 
typically pick the telemetry model, a dynamics simulator would 
pick the hardware model, and error analysis software would use the 
engineering model. The use of inheritance reduces the redundancy 
between the different applications, which saves effort in both 
development and maintenance. 

The other area in which the simulator can be made more general is 
the module types packages. These packages are not defined as 
generic units, but they contain a mix of mission-specific 
parameters (such as default initial conditions) and mission 
independent parameters. The nesting of a generic package within 
the types package is one possible way to make the system easier to 
configure. The nested generic would be parameterized by the 
mission dependencies, with the rest of the types package remaining 
mission independent A library instantiation of the nested generic 
would then be created to define the module's use for a particular 
mission rather than to extensively modify the types packages. 
Figure 14 shows how this would work for a fine Sun sensor 
module. The cost of this is that the other packages in a module 
now need to import both the types package and the instance of the 
nested generic, whereas only the types package was needed before. 
The stria separation of the parameterized part from the consistent 
part is worth the added complexity. 



Figure 14. 


4. FUTURE DIRECTIONS 

The experiences of the UARSTELS and GENSIM projects have 
demonstrated that the Ada language, and particularly generics, can 
be used to produce verbatim reusable components that can be fit 
into more than one architecture. Some other Ada language features 
need to be examined more closely to see how useful they are for 
simulation software. There has been a trend to using more strong 
typing as more experience is gained, but the FDD's Ada software 
has not gone as far as the Common Ada Missile Packages 
(CAMP) packages in using distinct types. The CAMP packages 
use a separate type for each unit of measure, both in generic 
parameter lists and in nongeneric code [Herr, 1988]. The 
advantage of using this degree of strong typing is that the compiler 
is able to catch any dimensionally incorrect computation. The 
disadvantage is that overloaded operators need to be defined 
anywhere that two or more different types can be correctly used in a 
computation. A balance needs to be found between the extremes of 


CAMP and of using a single floating-point type as the basis of all 
calculations. To do this, criteria must be defined for the proper use 
of Adas typing features. When to use or not to use types, 
subtypes, derived types, or private types needs clear definition. 

This paper's discussion of inheritance focuses on nested generic 
instantiations as a means of implementing the concept. An 
alternate approach is to use derived types to simulate inheritance 
[Perez, 1988]. In the simulators discussed earlier, generics are used 
for both parameterization and for inheritance. To use derived types 
for inheritance would require the investigation of the interaction 
between parameterization and inheritance when different language 
features are used. 

General Concepts for Large-Scale Verbatim Reuse 

The lessons learned by the UARSTELS and the GENSIM projects 
have led us to a general reuse model This model defines different 
levels of reuse and which reuse- in- the -small techniques should be 
applied at which level. Figure 15 shows the leveled reuse model 
on the left and typical examples on the right As in most layered 
models, the higher layers depend on services provided by the lower 
layers. 
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The lowest layer of the model is the language extension layer . This 
layer’s purpose is to create a problem-specific language by adding 
reusable Ada components to the existing capabilities of the Ada 
language. In the flight dynamics domain, this means defining 
types and operations for mathematical constructs such as vectors, 
matrices, end orbits. Applications code can then be developed 
using the specialized capabilities rather than predefined Ada 
constructs. This level can be considered the state of the practice for 
software reuse. The Booch components and the EVB GRACE 
components are at this level. 

The language extension layer itself uses a layered approach. The 
domain-specific objects are usually built on top of more general 
objects. The orbit data described above is specific to flight 
dynamics, but it is represented as two vectors representing position 
and velocity. When carried into design, an orbit data types package 
would depend on a more general linear algebra package that exports 
vector types. 

The other important distinction at this level is between entity 
abstractions and action abstractions. An object with action 
abstraction is completely described by what it does. A sort 
package provides operations to sort data; a random number 
generator generates random numbers. An object with entity 
abstraction has attributes beyond its set of compulations. For 
example, a queue can be described as a set of homogeneous daia 
that is accessed and modified using a FIFO protocol 
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Figure 16 shows how the level of abstraction and level of 
generality can be used to characterize language extension 
components. Some typical simulator components arc characterized 
by these two characteristics. The scale from domain specific to 
general is more continuous than is shown on this diagram. For 
example, a linear algebra package is specific to the mathematical 
domain, but it is considered a general-purpose package in the flight 
dynamics domain. Thus, it would fall somewhere in the middle of 
the scale. The distinction between entity and action abstraction is 
more clear cut. If the object has relevant properties beyond the 
actions it performs, it has entity abstraction. These properties are 
seen in Ada code as state information that can be retrieved and 
modified by a package’s operations. 
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The next level of the model is the domain level . This is the level 
at which the major problem domain entities reside. State-of-the-art 
reuse Libraries such as the CAMP contain components that are 
reused at this level [Herr, 1988]. Both the domain level and 
language extension level consist of objects and classes. The 
difference is that the objects at the domain level define the problem 
domain, and the objects at the language extension level are a 
means of expressing the model for a given problem domain object 
The fine Sun sensor abstract data types described in the previous 
section are all problem domain entities. They are described in 
terms of vector and matrix algebra, and in terms of standard error 
sources, telemetry encoding, and sensor failure models. The 
generic packages for fine Sun sensor data types implement the 
domain entities using capabilities provided at the language 
extension leveL 

Figure 13 shows how a mix of domain entities and generic 
language extensions can be used to build a hierarchy of classes and 
subclasses. The measurement utilities, hardware utilities, and the 
telemetry utilities are all language extensions, but they are used in 
building the problem domain inheritance model. 

The next level of reuse is the component template level, the level 
at which generic components are built to fit into a given system 
architecture. The GENSIM SHEM modules and the UARSTELS 
sensor models are examples of component templates. Components 
can be built directly from problem domain objects, or they can 
provide indirect support. In GENSIM, the SHEM modules will be 
built around abstract data types, such as those provided for the fine 
Sun sensors, and the standard module database and module results 
packages that are instantiated to support the module. In addition to 
these packages, a standard screen format file is used by the user 
interface to allow user inputs for each SHEM module. The key 
distinction is that the component template level defines all the 
components needed to fit a problem domain object into a given 
system architecture, where the domain level consists of a set of 
objects that are not constrained by & particular system design, but 
only by the problem being solved. 


The component template level objects arc also parameterized, but 
U)c emphasis shifts somewhat. The fine Sun sensor abstract data 
types are parameterized by data types for vectors and matrices and 
by operations needed to interface with other problem domain 
objects. The fine Sun sensor module is parameterized by items 
such as the number of sensors, the default input values, and 
selections of which inputs a user is allowed to modify. Some 
values of problem domain parameters may be constrained at this 
level. Figure 15 gives the example of Double_PTecision_FSS_ 
Module. This module has been constrained to use a particular 
floating-point data type, but it is still parameterized by the 
number of sensors and default values. 

The top level is the system template level. A generic system is a 
reusable design into which individual components can be fit. 
Objects at this level are parameterized by the set of components 
being used in a particular configuration and by any other values 
that have a system-wide effect. In GENSIM, the parameterization 
of the generic case interface is related to the particular set of SHEM 
modules being used. The simulation executive is parameterized 
both by the set of components being used and by the spacecraft’s 
control modes, which affect how often these components need to be 
executed. 

The two template levels provide the capability of quickly building 
a software system. Like the language extension and domain levels, 
the capabilities provided by the lower level are used by the higher 
one. The key distinction is that the lower two levels give a 
complete definition of the problem domain, and the upper two 
levels give a complete definition of a generalized software system 
architecture. It is important that the problem domain objects be 
completely independent of particular system architectures. To 
achieve this, the lower two levels from Figure 15 are grouped as 
problem domain levels and the upper two are grouped as 
architecture levels. 

The discussion in this section has focused on design issues, not 
how Ada should be used to realize these designs. The principles 
that apply to reuse in the small can be extended to reuse in the 
large. A developer must still be concerned about a mix of generic 
packages and their instantiations, and the coupling between 
components remains a key issue. 

In the problem domain level, the only coupling between objects 
should be defined by the problem. The preferred means of linking 
objects together is to restrict dependencies to those between library 
instantiations. One previously mentioned exception to this is the 
simulation of inheritance. Other relationships can also be 
simulated through nested generic instantiations or nested generic 
declarations. An example where nested instantiations are useful is 
in the case where one object is built from simpler components, as 
an inertial reference unit (IRU) is built from gyroscopes. The IRU 
presents a somewhat different interface than a gyroscope, although 
they are strongly related. The nested generic declarations are useful 
when alternate models depend on the same objects or types. For 
example, an orbit types package is parameterized in terms of 
simple mathematical functions, but they are used by a variety of 
different models for propagating orbits over time. Rather than 
nesting instantiations of the orbit types within several different 
models, the designer can present the models as a set of options 
that depend on the same orbit types. 

Importing other library uniLs into generic units is not a problem 
when used for component templates or system templates. Figure 
17 shows where the generic case interface package imports the 
instantiation of a generic types package. This interface types 
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package provides standard data types for communication between 
the user, the stored simulation data, and the SHEM modules. The 
designer should try to minimize this sort of coupling. In 
GENSIM, only interface types and a common types package are 
imported into generics in this manner. 
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5. MEASURING THE EFFECT OF LARGE-SCALE 
VERBATIM SOFTWARE REUSE 

This section discusses the impact of the verbatim reuse on project 
management by describing how costs are affected and the effects of 
the layered model. A recent SEL study [Solomon, 1987] 
characterized software components as being new, rebuilt (greater 
than 25-percent modification), adapted (up to 25-percent modified), 
and verbatim (unmodified). Expressed as a percentage of the cost of 
a new component, the costs of the different types of reused 
components are approximately : 


Verbatim 

10% (actually 7.2 %) 

Adapted 

30% 

Rebuilt 

50% 


To make conservative estimates, the 10-percent figure is used for 
verbatim components, and any nonverbatim component is assumed 
to be new. 


levels translate to a cosi savings of from 60 to 70 percent over an 
all-new system or at least 50 percent from current systems. 

The key to achieving high levels of verbatim reuse is to reuse 
specifications and design. The analysts who define the 
requirements for FDD systems developed common mathematical 
specifications for all systems supporting EUVE and UARS. The 
current estimate for EUVETELS reuse from UARSTELS is 87 
percent, which translates to approximately 80-perccnt cost savings 
over a new system. Even the FORTRAN software supporting 
EUVE has a reuse level of from 60 to 70 percent from UARS, 
whereas typical levels fall into the 20- to 30-percent range. The 
increase from reusing the mathematical specifications is much 
greater than the increase observed as the result of using Ada as the 
implementation language for simulators fBrechbiel, 1989]. These 
data confirm the correctness of GENSIM’s use of a set of standard 
mathematical specifications. 

In addition to measuring the level of verbatim reuse, the effect of 
verbatim reuse can be divided into the reuse of problem domain 
components and the reuse of components at the architecture levels. 
No FDD simulators have been developed using the proposed reuse 
model, so the estimate will be based on the fact that the user 
interface for the dynamics simulators typically contains 40 percent 
of the source lines of code and no problem domain objects. Since 
the capabilities of the GENSIM simulation executive and case 
interface subsystems are currently distributed among other 
subsystems, 40 percent is a conservative estimate. It is probably 
correct to assert that the benefits of reusable architectures equal or 
exceed those of developing reusable problem domain components. 
It is clear that these benefits are roughly equal. 


6. MANAGEMENT RECOMMENDATIONS 

The primary management recommendation is to build the problem 
domain levels first and to build them bottom up. The language 
extension layer is a means of expression for domain objects and 
classes. The domain objects and classes serve as the building 
blocks for reusable system architectures. Another advantage of 
building the problem domain layers first is the ability to build 
multiple architectures from the same set of problem domain 
objects. For simulation applications, this means that the same 
set of problem domain objects could be used to build a dynamics 
simulator, a telemetry simulator, or a combined dynamics and 
telemetry simulator. 

The strict separation of problem layers from architecture layers also 
provides the means of keeping up with technology. The same 
domain objects would be usable on either an 8086 based computer 
with a monochrome text screen or on an 80386-bascd computer 
with high-resolution graphics. The architecture of the system 
would be changed, although it would probably not be rebuilt from 
scratch. The architecture of a system should be driven by 
technology, and the solution of flight dynamics problems should 
not be. The separation of these considerations in design makes it 
easier to manage technological change. 


The GENSIM cost study [Mendelsohn, 1988] shows that the 
current levels of reuse for dynamics simulators save 15 to 20 
percent over all-new systems. The study also determined that 
dynmrics simulators have a potential for about 70- to 80-pcrccnt 
verbatim reuse; only the spacecraft control system code is 
developed from scratch for each mission. These verbatim reuse 


7. CONCLUSIONS 

The current state of the art in software reuse is to provide problem 
domain components and problem domain objects. This paper has 
demonstrated that designing verbatim reusable components at the 
architecture level can create approximately the same savings as the 
current staLe of the art. The new approach that needs to be applied 
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to future systems is to strictly separate the problem domain objects Stark, M.. and E. Seidewitz, Towards a Genera] Object-Oriented 

from the particular system architectures and to build the problem Ada Life Cycle," Proceedings of the Joint Ada Conference , March 

domain layers from the bottom-up. When this approach is used to 1987 

develop verbatim reusable software, the resources saved can be 

applied to new problems (extending the problem domain) or to 

provide better solutions to existing problems by upgrading the 

architecture. 
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